Protect production sources
Read boundaries, limits, cancellation, replica routing and credential storage.
BytesMap separates ordinary reads from explicitly authorized writes. Query limits, read-only sessions, scoped credentials and current resource permissions reduce risk; they do not guarantee that any query is safe for any production database. Use a dedicated database account and size its limits for the source's capacity.
This guide describes the implemented boundaries. Managed replication, automatic test-environment provisioning and a distributed query budget are not implemented.
Reads and writes
Ordinary SQL queries, chart data and query plans use the read path. Saving a definition or arranging a dashboard does not run it. Reading source data requires execution access; authoring access alone does not grant it. Published dashboards and workflows execute through their validated publication and permission rules. HTTP, SDK, CLI and MCP use the same platform operations.
Writes do exist. Supported workflow mutation nodes use a separate adapter transaction path with table/column checks and current authorization. Reader-role provisioning and event ingestion are also explicit write operations. These paths must not be confused with the normal query runner. A writable source credential does not itself grant a BytesMap user permission to mutate data. See Identity and access and Workflow automation.
Prefer database accounts restricted to the schemas, tables and operations a workspace needs. Do not use an administrator login for ordinary reporting. The database's grants remain the final boundary for functions and capabilities that the application cannot fully analyze.
Statement validation
The shared SQL guard tokenizes comments, strings, quoted identifiers and parameter
references before checking statement separators and keywords. It accepts a single
supported read statement and rejects write verbs, transaction control and
executing EXPLAIN ANALYZE. Each adapter further limits the statements it accepts.
This is a lexical guard, not a full SQL grammar or a proof that a function has no side effects. Read-only sessions and least-privilege database accounts are still necessary. Named query values use adapter parameter binding; callers must not construct SQL by concatenating untrusted values. Configurable identifiers pass through the adapter's identifier handling, not value interpolation.
| Source | Ordinary read boundary |
|---|---|
| PostgreSQL | New pool connections start with default_transaction_read_only=on. Query timeouts are applied on the session; tighter operation deadlines use a read-only transaction with local settings. The health check separately reports whether the credential has write grants. |
| MySQL / MariaDB | Every connection is initialized with a read-only session transaction mode. Queries receive the engine's supported execution-time limit. |
| SQLite | The adapter uses the read-only query path and checks query_only; execution is isolated from the application event loop. |
| MongoDB | The adapter permits its supported read/aggregation stages and uses cursor limits and deadlines. The health check only identifies known read-only roles; unknown/custom privileges are not certified safe. |
| Redis | Only supported read commands are exposed. Redis has no equivalent session read-only mode, and the health check does not certify the account's ACL. Configure a restricted Redis ACL. |
| ClickHouse | The adapter checks the server's read-only profile and applies query/result settings. A readonly=2 profile allows those per-query settings. |
Query limits
Open a source's connection settings and use Query safety to set its ceilings. The contracts, editor and connection layer share the same bounds:
| Setting | Default | Allowed range |
|---|---|---|
| Query timeout | 10 seconds | 1–300 seconds |
| Returned rows | 50,000 | 1–1,000,000 |
| Result data | 32 MiB | 1 KiB–1 GiB |
| Concurrent connections | 4 | 1–32 |
An individual operation may use a lower budget. PostgreSQL, MySQL and MariaDB include pool checkout, query execution and result transfer in the operation deadline. Reads add an outer row limit with one extra row to detect truncation. Their incremental row readers enforce the UTF-8 JSON size of the returned data array, including separators and escaping, before retaining another row. The extra truncation row is not part of that returned array. Result metadata and transport headers are not included in this data-byte limit.
The connection limit applies to a source's execution pool in one process. A primary write pool and replica read pool are separate; API/worker processes also have separate pools. PostgreSQL and MySQL/MariaDB can open one additional control connection per pool to cancel its own running statements. Allow for these connections when setting database-wide quotas. This is not a global tenant concurrency budget.
On timeout or cancellation, the SQL adapters discard the affected data connection
and request server cancellation through a bounded control connection:
pg_cancel_backend for PostgreSQL and KILL QUERY for MySQL/MariaDB. If that
request cannot reach the server, the configured server-side deadline remains the
fallback. A client abort alone is not evidence that the server has already stopped.
The pool remains usable after canceled or failed work; writes are not automatically
retried because their outcome may be uncertain.
Other engines use their own cursor, command or HTTP limits. Redis commands that have already started cannot be individually canceled by this adapter. A result cap limits returned data, not scanned rows, CPU, temporary files or aggregate work. A driver may allocate an individual large field before the row cap can reject it. Keep database-level statement/resource limits and monitor the source under real load. Query cost distinguishes plans, measured execution and application observations; it is not an admission-control guarantee.
API request budgets
With REDIS_URL configured, API replicas share fixed-window request budgets through
Redis. Use the same BYTESMAP_RATE_LIMIT_NAMESPACE and BETTER_AUTH_SECRET on every
replica of one deployment; use a different namespace for unrelated deployments
sharing a Redis database. The root SaaS Compose profile configures this shared
backend. Namespace or key-secret changes start a different budget, so do not vary
them per replica. Keys contain a hashed namespace and keyed hashes of the policy
and subject, rather than raw user IDs, organization IDs or client addresses.
SQL execution, profiling and MCP transport budgets remain per organization; ordinary authenticated reads and writes remain per user. Anonymous and sign-in budgets use the verified transport client address. Proxy headers count only when the immediate peer matches the configured trusted proxy policy.
Each Redis operation atomically updates one expiring counter. Denied requests do
not extend the window. Redis supplies the expiry and reset time; an API replica's
clock cannot reset another replica's budget. Exhausted budgets return 429 with
Retry-After and the existing X-RateLimit-* headers. If configured Redis is
unavailable, slow, or not yet connected, requests return 503 (rate.unavailable)
with a short retry delay. There is no fallback to an independent local allowance,
offline command queue, or replay of an uncertain counter update. The API owns and
closes its Redis connection explicitly. Its liveness endpoint alone does not prove
that Redis-backed operations are ready.
Without REDIS_URL, budgets remain process-local for desktop and single-host use.
Running multiple such API processes multiplies the effective request ceiling.
Redis eviction, lost persistence, or administrative counter deletion can also
reset a budget: use a dedicated, appropriately sized Redis deployment and monitor
its availability and memory. These are HTTP request budgets, not distributed
source-connection limits, a database CPU quota, or a per-query cost estimate.
Read replicas
Read destination → Read replica configures an existing replica with separate credentials. It does not create replication, clone a database or monitor lag. Ordinary reads use that target; authorized writes always use the primary. A replica error is returned to the caller: there is no silent fallback that sends reporting load to production's primary.
For SSH sources, configure the replica's host and port as reached from the server. The replica does not reuse the primary's forwarded destination. Pool identity includes the organization, source, execution purpose and connection configuration; credential or policy changes retire the old connections without merging tenants or routing writes to a replica.
Replica lag can make a read after a successful write temporarily stale. Choose primary reads when immediate consistency is required, or design the workflow to tolerate lag. Verify schema compatibility and replication health outside BytesMap.
Transport and credentials
Use TLS with certificate/hostname verification for remote databases. Encryption without verification does not establish the server's identity. An SSH tunnel protects its SSH leg; a database on another host behind the bastion may still need TLS for the remaining network leg. The source details show the configured transport rather than treating every tunnel as end-to-end database TLS.
Saved credentials use the host's SecretCodec. The API's default codec is
AES-256-GCM with a random nonce and authenticated ciphertext:
- With
BYTESMAP_MASTER_KEY, new values use a versioned envelope derived from that key. Cloud mode requires it. Use a generated, independent high-entropy key. - Existing unversioned values still use the legacy
BETTER_AUTH_SECRET-derived key. Local installations without a master key retain that legacy format. - Unknown formats, missing keys, wrong keys and modified ciphertext fail closed. The reader does not silently try another encryption key.
- A native host can supply its own credential codec. Never put a key or plaintext database credential in a browser renderer.
Back up the metadata database and the applicable encryption keys separately. API and worker processes must share the same keys. Adding a master key does not automatically re-encrypt old credentials: preserve the previous auth secret until all legacy credentials have been replaced/re-saved and verified. Rotating the auth secret leaves new master-key envelopes readable but can invalidate sessions and legacy credentials. Rotating the master key without re-encryption makes its existing envelopes unreadable. There is no automatic rekey migration yet.
Vault resolution is organization-scoped. Cloud requests cannot resolve arbitrary host environment-variable secret references; trusted local deployments may use them. Normal source responses omit plaintext credentials, and public connection metadata is redacted. Keep exports, query literals and diagnostic attachments free of secrets too; encryption at rest does not sanitize data a user puts in SQL.
Access, agents and data
Use authenticated deployment mode for a shared instance. The guarded anonymous mode is only for explicitly enabled local device access; it is not a substitute for authentication behind a public proxy. See the deployment matrix in Identity and access.
Organization membership, resource grants and delegated credential scopes are checked by the shared platform. Project navigation and browser preferences cannot create authority. Embedded applications remain responsible for their own row-level business authorization; a BytesMap project is not a source database's row-level tenant boundary.
Authorized MCP inspection and execution can return real database values. An agent/model provider may receive those values. A permitted workflow can also perform declared effects. Limit agent grants to the needed resources and actions, and treat returned text as data, not trusted instructions. See MCP.
Validation and remaining limits
Isolated integration tests cover real PostgreSQL, MySQL and MariaDB query cancellation, result limits, pool recovery and primary/replica separation. The adapter suite also covers engine-specific read restrictions and error handling. These tests establish the tested behavior, not a guarantee for every server version, proxy, network failure or workload. Use a staging source and inspect its resource usage before publishing expensive dashboards or frequent schedules.
The local source soak repeatedly exercises the shared source operations with real PostgreSQL metadata and source sessions. Its report records concurrency, cancellations, recovery rounds, timing retention and post-close session cleanup. The fixture uses its own temporary PostgreSQL process; its synthetic local timings do not establish production capacity or a global connection budget across separate API and worker processes.
The soak's --transport http mode uses two real API processes with shared
PostgreSQL metadata and an owned Redis budget store. It checks authenticated
reads, tenant boundaries, live credential revocation, cancellation, bounded
source sessions, shared rate limits and failure when Redis is unavailable.
Requests keep their normal production limits. Two API processes still own two
separate source pools; Redis request budgets do not combine those pools.
Report exploitable issues through the repository's private vulnerability reporting, as described in CONTRIBUTING.md.