Developer documentation

How to point each protocol's official SDK at the public service, what works, what differs from the real service, and what every refusal means.

Protocol guides

• AMQP 0-9-1

  RabbitMQ client libraries over amqps on port 5673. Python, Node.js, Go, Java.

• Amazon SQS

  AWS SDKs and the AWS CLI. Python, JavaScript, Go, CLI.

• Google Cloud Pub/Sub

  Google Cloud client libraries over gRPC. Python, Node.js, Go.

• Azure Queue Storage

  Azure Storage SDKs, path-style. Python, JavaScript, .NET.

• Azure Service Bus

  Azure Service Bus SDKs with a connection string. Python, JavaScript, .NET.

Endpoints

Every endpoint uses TLS with a publicly trusted certificate; there are no plain-text endpoints. Your firewall must allow outgoing connections to port 443, and to 5671 for Service Bus or 5673 for AMQP 0-9-1.

Tenant identifiers

• Exactly 24 characters: a lowercase ASCII letter, then 23 lowercase ASCII letters or digits. As a regular expression: ^[a-z][a-z0-9]{23}$.
• Choose it at random. The service can check the format, not the randomness, and a memorable identifier is a guessable one.
• The first request naming a valid identifier creates its tenant. One identifier is one tenant in every protocol.
• Anything else is refused, including empty values and SDK placeholders such as guest, test, test-project, devstoreaccount1 and RootManageSharedAccessKey. There is no default tenant.

To generate one from a shell:

python3 -c 'import secrets, string; a = string.ascii_lowercase; print(secrets.choice(a) + "".join(secrets.choice(a + string.digits) for _ in range(23)))'

Entities need not be created

Using a queue or topic name creates it, in every protocol, and entities created that way are deleted after an hour empty and unused. Pub/Sub's rule: publishing to topic X creates subscription X, and pulling X~name creates a further subscription to X. How it works has the details for each protocol.

Limits and their errors

Each limit is refused with the protocol's own error, which the SDK raises as it would against the real service. Values above a limit are refused, never silently reduced.

The error for each limit

Status codes are the HTTP status each SDK sees. Names in brackets are the Python SDK's exception classes.

Troubleshooting

"this public service needs a tenant identifier…"

The full message is:

this public service needs a tenant identifier in your SDK's credential field: a lowercase letter followed by 23 lowercase letters or digits (24 characters), chosen at random; the service's web page generates one

The field that names the tenant does not hold a valid identifier. Check, for your protocol:

• AMQP 0-9-1: the URL ends in /<identifier>. An empty path or / is the default virtual host, which is refused. pika reports ProbableAccessDeniedError with (530) NOT_ALLOWED.
• SQS: the access key id is the identifier. Environment variables or ~/.aws/credentials may override what you think you set; aws configure list shows which is used.
• Pub/Sub: the project in the topic or subscription path is the identifier. A project picked up from GOOGLE_CLOUD_PROJECT or gcloud's configuration is not.
• Azure Queue Storage: the account URL's first path segment and the credential's account name are both the identifier, and must be the same.
• Service Bus: SharedAccessKeyName is the identifier. Python raises ServiceBusAuthenticationError; the message is in its inner error.

Uppercase letters, hyphens, a leading digit or the wrong length are all refused.

"too many new tenants; try again later or reuse your identifier"

Your address created 20 tenants within the last hour, or the service holds its maximum of 10,000. Reuse one identifier for all your work: one tenant can hold 100 entities in every protocol.

Throttling

More than 100 requests a second from one address (after a burst of 200) are answered with the protocol's throttling error: SQS RequestThrottled, Pub/Sub RESOURCE_EXHAUSTED, Azure ServerBusy, Service Bus server-busy. The SDKs treat these as retryable and back off on their own, so you usually see slower calls rather than errors. If you turned retries off, retry with back-off yourself. Batch operations (SQS SendMessageBatch, Pub/Sub's batching publisher, Azure's 32-message gets) do more per request.

An address that keeps exceeding the limits, refused 100 times within a minute, is banned for a while: a minute the first time, then 10 minutes, an hour, and a day each time after, until it has gone a day without a ban. While banned, its HTTP requests are answered 429 Too Many Requests with Retry-After (Pub/Sub RESOURCE_EXHAUSTED), and its AMQP connections are closed at once. Client libraries that back off after a throttling error never get near this.

Long-lived streams count each request on them: a Pub/Sub streaming pull counts each acknowledgement request, and a Service Bus connection counts each message sent and each link attached.

Size errors

The size limit counts the body plus attributes, headers or properties. At 256 KiB of body you are already over if you add any attributes. Azure Queue Storage measures the message text after any encoding your SDK applies (base64 grows it by a third) against 64 KiB. AMQP 0-9-1 is strictest: an oversized message closes the whole connection with 311 CONTENT_TOO_LARGE. The broker advertises its limit as max_message_size in its connection properties.

Queue full

A queue or subscription holding 1,000 messages, or a tenant holding 16 MiB of messages across its queues (counting messages received but not yet acknowledged), refuses further sends until something is consumed. Pub/Sub's publisher retries RESOURCE_EXHAUSTED, so a publish to a full subscription can wait until its deadline. AMQP 0-9-1 publishers only learn of the refusal with publisher confirms on (confirm.select).

My queue or messages disappeared

• Your tenant had no activity for 7 days and was deleted with everything in it.
• Data was lost in maintenance. It normally survives restarts, but nothing is backed up or guaranteed.
• The queue was created implicitly and stayed empty and unused for an hour; create it explicitly to keep it.
• The messages expired: nothing lives longer than 1 hour.
• You are using a different identifier from before (this site keeps one per browser).

Pub/Sub pull answers NOT_FOUND

A pull names only a subscription, so an unknown name cannot be created. Pull subscription X after publishing to topic X, pull X~something, or create the subscription with CreateSubscription.

Connection or TLS errors

• Do not set PUBSUB_EMULATOR_HOST or a Service Bus UseDevelopmentEmulator=true: both turn TLS off, and the public endpoints speak only TLS.
• AMQP 0-9-1 needs amqps:// and port 5673. Service Bus needs outgoing port 5671; the AMQP-over-WebSockets transport the SDKs offer is not supported.
• A connection closed immediately on an AMQP port means your address holds 50 connections already, opened more than 5 a second (after a burst of 20), is temporarily banned, or the service is down. Check the front page's statistics.