Skip to main content

rfd--dashboard

Overview​

  • Namespace: rfd--dashboard
  • Purpose: Referring Doctor Dashboard Backend - PRODUCTION
  • Age: ~2 years 174 days (since April 2023)
  • Status: x Partially Active - Dashboard API active, consumer and cron scaled to 0
  • Workloads: 3 deployments (1 active, 2 scaled to 0)
  • Environment: PRODUCTION - Doctor dashboard and analytics

Architecture​

Referring doctor dashboard system with minimal active components:

  • Main Application: REST API backend (1 replica) - Active
  • Event Consumer: Recent order processing - x Scaled to 0
  • Scheduler: Cron jobs - x Scaled to 0

Auto-Scaling Configuration​

No Auto-Scaling Configured:

  • No HorizontalPodAutoscalers (HPAs)
  • No KEDA scaled objects
  • Fixed replica counts

Workload Categories​

Main Application (1 deployment - Active)​

NameReplicasStatusPurpose
rfd--dashboard--be--app--prod1/1RunningMain RFD dashboard API

Event Consumer (1 deployment - Scaled to 0)​

NameReplicasStatusPurpose
consumer-recent-order0/0x Scaled to 0Recent order processing (inactive)

Scheduler (1 deployment - Scaled to 0)​

NameReplicasStatusPurpose
cronjob0/0x Scaled to 0Scheduled cron jobs (inactive)

Services​

NameTypeCluster IPPortsNodePortPurpose
rfd--dashboard--be--app--prodNodePort10.8.31.628030591Main RFD dashboard API

Access & Management​

View all resources:​

kubectl get all -n rfd--dashboard

Check main application:​

kubectl get pods -n rfd--dashboard | grep "app--prod"
kubectl logs -f deployment/rfd--dashboard--be--app--prod -n rfd--dashboard

Check scaled-to-0 deployments:​

# View scaled deployments
kubectl get deployments -n rfd--dashboard | grep "0/0"

# Scale up if needed
kubectl scale deployment rfd--dashboard--be--consumer-recent-order--prod -n rfd--dashboard --replicas=1
kubectl scale deployment rfd--dashboard--be--cronjob--prod -n rfd--dashboard --replicas=1

Restart main app:​

kubectl rollout restart deployment/rfd--dashboard--be--app--prod -n rfd--dashboard

Monitoring​

Resource usage:​

kubectl top pods -n rfd--dashboard --sort-by=memory
kubectl top pods -n rfd--dashboard --sort-by=cpu

Deployment status:​

# All deployments
kubectl get deployments -n rfd--dashboard

# Active only
kubectl get deployments -n rfd--dashboard | grep -v "0/0"

Events:​

kubectl get events -n rfd--dashboard --sort-by='.lastTimestamp' | head -20

Data Flow​

Doctor Dashboard Request
    ↓
rfd--dashboard--be--app--prod (NodePort 30591)
    ↓
Main RFD Dashboard API (1 replica)
    ↓
Database (external)
    ↓
Dashboard analytics, reports, order views

Note: Consumer and cron jobs scaled to 0 (inactive)

Dashboard Workflow​

1. Dashboard API (Active)​

  • Doctor dashboard views
  • Analytics and reports
  • Order history display
  • Performance metrics
  • API serving dashboard frontend

2. Recent Order Consumer (Scaled to 0)​

  • Inactive - scaled to 0
  • Previously processed recent order events
  • Review if still needed or permanently deprecated

3. Cron Jobs (Scaled to 0)​

  • Inactive - scaled to 0
  • Previously handled scheduled tasks
  • Review if still needed or permanently deprecated

Production Considerations​

High Availability​

x Single Points of Failure:

  • Main API: 1 replica (no HA)
  • Only 1 active deployment in entire namespace

x Scaled to 0 (67% of deployments):

  • consumer-recent-order: Scaled to 0
  • cronjob: Scaled to 0
  • Cleanup Opportunity: Review and remove if permanently unused

Recommendations​

  1. Main API Resilience:

    • Currently 1 replica (single point of failure)
    • Increase to 2+ replicas or add HPA
    • Critical for doctor dashboard access

  2. Review Scaled-to-0 Deployments:

    • consumer-recent-order: Scaled to 0 (~2 years)
    • cronjob: Scaled to 0 (~2 years)
    • If permanently unused, remove deployments
    • If needed, scale up and add monitoring

  3. Add Auto-Scaling:

    • Consider HPA for main API
    • Scale during peak dashboard usage times

  4. Cleanup Checklist:

    • Confirm with RFD team if consumer/cron still needed
    • Verify no plans to re-activate
    • Check if functionality moved elsewhere
    • Remove deployments if confirmed unused
    • Update architecture documentation

  5. Monitoring Priorities:

    • API response times
    • Dashboard load performance
    • Database query performance

Troubleshooting​

Main API issues:​

# Check API pod
kubectl get pods -n rfd--dashboard | grep "app--prod"

# Check logs
kubectl logs -f deployment/rfd--dashboard--be--app--prod -n rfd--dashboard --tail=100

# Test API endpoint
kubectl port-forward -n rfd--dashboard service/rfd--dashboard--be--app--prod 8080:80
# Access http://localhost:8080

Scaled-to-0 investigation:​

# Check deployment history
kubectl rollout history deployment/rfd--dashboard--be--consumer-recent-order--prod -n rfd--dashboard
kubectl rollout history deployment/rfd--dashboard--be--cronjob--prod -n rfd--dashboard

# View deployment details
kubectl get deployment rfd--dashboard--be--consumer-recent-order--prod -n rfd--dashboard -o yaml
kubectl get deployment rfd--dashboard--be--cronjob--prod -n rfd--dashboard -o yaml

# Scale up for testing (if needed)
kubectl scale deployment rfd--dashboard--be--consumer-recent-order--prod -n rfd--dashboard --replicas=1
kubectl scale deployment rfd--dashboard--be--cronjob--prod -n rfd--dashboard --replicas=1

# Watch startup
kubectl get pods -n rfd--dashboard -w

Performance issues:​

# Check API resource usage
kubectl top pods -n rfd--dashboard

# Check API logs for slow queries
kubectl logs deployment/rfd--dashboard--be--app--prod -n rfd--dashboard --tail=200 | grep -i "slow\|timeout\|query"

# Restart API
kubectl rollout restart deployment/rfd--dashboard--be--app--prod -n rfd--dashboard

Performance Metrics​

Current Scale​

  • Main API: 1 replica (no HA)
  • Consumer: 0 replicas (scaled to 0)
  • Cron: 0 replicas (scaled to 0)
  • Total Active Pods: ~1 pod
  • Scaled to 0: 67% of deployments (2 of 3)

Stability​

  • Namespace Age: ~2 years (mature)
  • Recent Updates: 382 days ago for app (very stable)
  • Scaled-to-0 Components: ~2 years (likely deprecated)

Cleanup Opportunity​

x 67% of deployments scaled to 0 - Large cleanup opportunity:

  • consumer-recent-order: Scaled to 0 since creation
  • cronjob: Scaled to 0 since creation

Action Required: Review with RFD team to confirm if these deployments can be permanently removed.