8.0 KiB
psql Tips — Advanced Debugging, Performance & Safety
Part of the psql tips reference. See also: tips-workflows.md
Advanced techniques for debugging, performance tuning, and safe psql usage.
Table of Contents
Performance Tips
Large Result Sets
-- Don't load entire result into memory
\set FETCH_COUNT 1000
SELECT * FROM billion_row_table;
-- Use \copy instead of COPY for client-side operations
\copy huge_table TO '/data/export.csv' WITH (FORMAT csv)
Pipeline Mode for Batch Operations
Pipeline mode batches multiple queries into single network round trips:
\startpipeline
INSERT INTO logs (msg) VALUES ($1)
\bind 'entry 1' \sendpipeline
INSERT INTO logs (msg) VALUES ($1)
\bind 'entry 2' \sendpipeline
INSERT INTO logs (msg) VALUES ($1)
\bind 'entry 3' \sendpipeline
\getresults
\endpipeline
This sends all three INSERTs in one network round trip instead of three.
Script Execution
# Run in a single transaction (faster, and all-or-nothing)
psql -1 -f migration.sql mydb
# Multiple files sequentially
psql -1 -f 001.sql -f 002.sql -f 003.sql mydb
Debugging and Introspection
See What psql Sends to the Server
-- Echo all SQL commands
\set ECHO queries
-- See the SQL behind \d commands (incredibly useful for learning)
\set ECHO_HIDDEN on
-- Or in noexec mode (show but don't execute)
\set ECHO_HIDDEN noexec
-- Now run any \d command to see its SQL
\dt
\d users
Check Query Plans
-- Basic plan
EXPLAIN SELECT * FROM users WHERE email = 'test@example.com';
-- With actual execution time
EXPLAIN ANALYZE SELECT * FROM users WHERE email = 'test@example.com';
-- Detailed with buffer info
EXPLAIN (ANALYZE, BUFFERS, VERBOSE) SELECT ...;
-- JSON format for tooling
EXPLAIN (FORMAT JSON) SELECT ...;
Lock Analysis
-- Current locks waiting to be granted
SELECT * FROM pg_locks WHERE NOT granted;
-- Blocked sessions with their blockers (PostgreSQL 9.6+)
SELECT blocked.pid,
blocked.query,
pg_blocking_pids(blocked.pid) AS blocked_by_pids
FROM pg_stat_activity blocked
WHERE cardinality(pg_blocking_pids(blocked.pid)) > 0;
Safety and Best Practices
Always Set ON_ERROR_STOP in Scripts
Without ON_ERROR_STOP, a script continues even after errors, potentially leaving the database in an inconsistent state:
-- Top of every script
\set ON_ERROR_STOP on
Use Single-Transaction Mode for Migrations
# -1 wraps everything in BEGIN...COMMIT
# On error, the entire migration rolls back
psql -1 -f migration.sql mydb
Never Use PGPASSWORD in Scripts
# BAD: Password visible in process list, env vars
PGPASSWORD=secret psql -c "SELECT 1" mydb# GOOD: Use ~/.pgpass (manually edit to avoid shell history)
touch ~/.pgpass && chmod 600 ~/.pgpass
# Then edit ~/.pgpass and your add:
# hostname:port:database:username:password
# Example: localhost:5432:mydb:myuser:mysecret
psql -c "SELECT 1" mydb
Preview Before Executing
-- Dry-run a script to see what commands will execute (shows SQL, does NOT execute)
\set ECHO all
BEGIN;
-- Paste or review migration SQL here, then ROLLBACK instead of COMMIT
\i migration.sql
ROLLBACK;
-- Or use \gdesc to check result columns without executing
SELECT * FROM complex_view \gdesc
Use \copy Over COPY
\copy uses client permissions and filesystem. SQL COPY runs on the server and requires superuser or pg_read_server_files/pg_write_server_files roles. \copy transfers all data through the client/server connection, which is less efficient than SQL COPY for very large datasets. For bulk data transfer, prefer SQL COPY when server-side file access is available.
-- BAD (requires server-side file access)
COPY users TO '/tmp/users.csv' WITH CSV HEADER;
-- GOOD (uses client-side file access)
\copy users TO '/tmp/users.csv' WITH CSV HEADER
search_path Safety for Untrusted Users
If untrusted users have access to the database, remove publicly-writable schemas from search_path at session start:
SELECT pg_catalog.set_config('search_path', '', false);
Automatic LISTEN/NOTIFY Polling
Whenever a command is executed, psql automatically polls for asynchronous notification events generated by LISTEN/NOTIFY. This happens without any special configuration.
Gotchas and Common Mistakes
Semicolons in \copy
\copy does NOT end with a semicolon. It's a meta-command:
-- CORRECT
\copy users TO '/tmp/users.csv' WITH CSV HEADER
-- WRONG (psql interprets the semicolon oddly)
\copy users TO '/tmp/users.csv' WITH CSV HEADER;
Variable Substitution and SQL Injection
psql variables are simple text substitution. They are NOT parameterized queries:
-- The :'varname' form (single-quoted) escapes embedded single quotes, making it
-- safe against SQL injection for STRING VALUES in WHERE clauses:
\set name "Robert'); DROP TABLE students;--"
SELECT * FROM users WHERE name = :'name';
-- Expands to: WHERE name = 'Robert''); DROP TABLE students;--'
-- The '' is an escaped quote, so the entire value is a single string literal — NOT injected.
-- However, :'varname' is NOT safe for identifiers or unquoted contexts.
-- For identifiers (table/column names), use :"varname" (double-quoted form).
-- For the safest parameterized queries, use \bind:
SELECT * FROM users WHERE name = $1;
\bind 'Robert' \g
Transaction State After Error
After an error in a transaction block, all subsequent commands fail until ROLLBACK:
BEGIN;
INSERT INTO users (id) VALUES (1);
INSERT INTO users (id) VALUES ('bad'); -- ERROR
INSERT INTO users (id) VALUES (2); -- Also fails!
COMMIT; -- Also fails!
Use ON_ERROR_ROLLBACK to auto-savepoint:
\set ON_ERROR_ROLLBACK on
BEGIN;
INSERT INTO users (id) VALUES (1);
INSERT INTO users (id) VALUES ('bad'); -- ERROR, auto-rollback to savepoint
INSERT INTO users (id) VALUES (2); -- This works now
COMMIT;
Pattern Matching Uses Regex
The * and ? in \d commands are converted to regex (.* and .). Advanced regex like [0-9] works. Important differences from standard regex:
.is a schema/object separator, not any-char (use?for any single char)$is matched literally, not as an anchor (pattern must match whole name anyway)- Within double quotes, all special characters (
*,?, regex chars) are literal
-- These work as expected
\dt user* -- matches users, user_accounts, etc.
\dt user? -- matches users, user1, etc.
\dt user[0-9]* -- matches user1, user2, user123
-- If your table name contains special chars, use double quotes
\dt "table.with.dots" -- matches literally, dots not treated as separator
Connection String vs CLI Arguments
# These are equivalent
psql -h localhost -p 5432 -U admin -d mydb
psql "postgresql://admin@localhost:5432/mydb"
# But you can't mix freely — URI overrides individual flags
psql -h otherhost "postgresql://admin@localhost:5432/mydb" -- uses localhost, not otherhost
\i vs \ir
\i filename— resolves relative to the current working directory (where psql was started)\ir filename— resolves relative to the currently executing script's directory
For script portability, prefer \ir:
-- In /scripts/migrations/run_all.sql:
\ir 001_schema.sql -- resolves to /scripts/migrations/001_schema.sql
\ir 002_data.sql -- resolves to /scripts/migrations/002_data.sql
\o and Query Output
\o redirects query output, not meta-command output:
\o /tmp/output.txt
SELECT * FROM users; -- goes to file
\d users -- also goes to file
\echo 'hello' -- goes to STDOUT (not affected by \o)
\qecho 'hello' -- goes to /tmp/output.txt (affected by \o)