Confluent Cloud

Confluent Cloud is the premium managed Kafka service — full Kafka API compatibility plus Confluent-exclusive features (KSQL for stream processing, Schema Registry for schema governance, 200+ managed connectors, Flink integration). For agents: use standard Kafka client libraries (confluent-kafka-python) for producer/consumer workflows — same code works against Confluent Cloud as self-hosted Kafka. REST Proxy enables HTTP-based produce/consume without Kafka clients (useful for agents in HTTP-only environments). Management API creates topics, manages ACLs, and configures connectors. Free trial ($400 credit). Confidence is docs-derived.

Kafka Producer/Consumer API (via confluent-kafka-python): Producer({bootstrap.servers, sasl.mechanisms, security.protocol, sasl.username, sasl.password}). producer.produce(topic, key, value). Consumer({...}).subscribe([topic]). consumer.poll(timeout_ms). REST Proxy: POST /topics/{topic} produces messages ({records: [{key, value}]}). GET /consumers/{group}/instances/{instance}/records consumes messages. Schema Registry: POST /subjects/{subject}/versions registers schema. GET /schemas/ids/{id} retrieves schema by ID. Management API (Confluent Cloud API): POST /kafka/v3/clusters/{cluster}/topics creates topic ({topic_name, partitions_count, replication_factor, configs}). GET /kafka/v3/clusters/{cluster}/topics lists topics. Connectors: POST /connect/v1/environments/{env}/clusters/{cluster}/connectors creates connector.

API key + secret auth: two types — (1) Kafka cluster API key (for Kafka producer/consumer + REST Proxy): sasl.mechanisms=PLAIN, security.protocol=SASL_SSL, sasl.username={api_key}, sasl.password={api_secret}. (2) Cloud API key (for management API + Schema Registry): Authorization: Basic base64(api_key:api_secret). Keys from Confluent Cloud Console → API Keys. HTTPS enforced. Schema Registry: separate credentials (schema_registry.url + basic.auth.credentials.source). ACLs: configure topic-level access per API key.

Kafka exceptions: KafkaException (producer/consumer errors). Delivery callback: poll delivery.reports() for async produce confirmation. Consumer offset management: commit offsets after processing (at-least-once delivery). REST Proxy: standard HTTP status codes. Confluent Cloud uptime at status.confluent.io. Schema evolution: backward/forward compatibility enforced by Schema Registry (400 on incompatible schema). Topic auto-create: disabled by default on Confluent Cloud — pre-create topics via Management API. Lag monitoring: consumer group lag via Cloud Console or API. Connector failure: check connector status via GET /connectors/{name}/status.

docs.confluent.io/cloud covers Kafka client setup (all languages), REST Proxy reference, Schema Registry API, Management API, Connector documentation, and KSQL guide. Getting started: Confluent Cloud free trial, create cluster, create API key — first message produced in under 10 minutes. confluent-kafka-python: pip install confluent-kafka. Community via Confluent Community Forum (community.confluent.io) and GitHub. Excellent documentation for a mature managed Kafka service.