data-streaming-consumers

Overview

Data Streaming Consumer Namespaces (5 namespaces):

Namespace	Purpose	Status
data-streaming-airtable-consumer	Airtable data sync	Active
data-streaming-hrservice-integration	HR service integration	Active
data-streaming-iris-consumer	IRIS system integration	Active
data-streaming-ns-rt-consumer	NetSuite real-time consumer	Active
data-streaming-veeva	Veeva integration	Active
data-streaming-veeva-consumer	Veeva consumer	Active

Common Architecture

All data-streaming consumers follow similar pattern:

• Consumer Service: Event consumer pods (variable replicas)
• Purpose: Process events from data streams
• Integration: Connect external systems (Airtable, HR, IRIS, NetSuite, Veeva)
• Data Flow: External system → Kafka → Consumer → Processing

Key Details

Airtable Consumer

• Syncs Airtable base data to internal systems
• Real-time sync for collaborative records
• Status: Active

HR Service Integration

• Integrates HR system data
• Employee and payroll data sync
• Status: Active

IRIS Consumer

• Radiology/imaging system integration
• Medical image data streaming
• Status: Active

NetSuite RT Consumer

• Real-time NetSuite data synchronization
• Financial data sync
• Status: Active

Veeva Consumers (2)

• Vault/Veeva system integration
• Document and content management
• Status: Active

Production Considerations

Generally Active:

• All consumers actively running
• Event-driven architecture
• Multiple integrations

Check Individual Status:

• Monitor for consumer lag
• Verify event delivery
• Check for processing errors

Recommendations

1. Verify replica counts for HA
2. Monitor queue depths and lag
3. Test failure scenarios
4. Document SLAs per consumer
5. Plan scaling strategy

Access

# Check specific consumer
kubectl get all -n data-streaming-airtable-consumer
kubectl logs -f deployment -n data-streaming-airtable-consumer

# Monitor consumers
kubectl top pods -n data-streaming-airtable-consumer