Monitoring tahlili (Observability)

Observability ichiga quyidagilar kiradi: Structured logging, Logback/Log4j2, MDC, Distributed tracing, OpenTelemetry, Zipkin, Metrics with Micrometer + Prometheus, Grafana dashboards, Alerting strategy.

1. Observability nima?

Observability — production’da ilova ichida nima bo‘layotganini tashqaridan turib tushuna olish.

Oddiy qilib:

App sekinlashdi, xato berdi yoki request yo‘qoldi. Siz log, metric va tracing orqali sababini topa olishingiz kerak.

Observability uchta asosiy ustunga tayanadi:

Logs
Metrics
Traces

Bular alohida-alohida emas, birga ishlaganda kuchli bo‘ladi.

2. Monitoring vs Observability

Monitoring

Monitoring — oldindan belgilangan narsalarni kuzatish.

Masalan:

CPU 90% bo‘ldimi?
RAM to‘ldimi?
API latency oshdimi?
Error rate ko‘paydimi?

Monitoring sizga “muammo bor” deydi.

Observability

Observability esa “muammo nega bo‘ldi?” degan savolga javob beradi.

Masalan:

/order endpoint sekinlashdi
 ↓
tracing ko‘rsatdi: Payment Service timeout
 ↓
metric ko‘rsatdi: payment_client_latency oshgan
 ↓
log ko‘rsatdi: connection pool exhausted

Monitoring signal beradi. Observability sababni topishga yordam beradi.

3. Nega Senior Java uchun muhim?

Junior yoki Middle developer ko‘pincha localda debugging qiladi.

Senior esa production’da ishlayotgan tizimni ko‘radi:

100 ta pod
20 ta microservice
Kafka
Redis
PostgreSQL
External payment API
Load balancer
Kubernetes

Bu joyda System.out.println() bilan muammo topib bo‘lmaydi.

Senior developer production muammoni quyidagicha tahlil qiladi:

Alert keldi
 ↓
Dashboard ochildi
 ↓
Error rate/latency tekshirildi
 ↓
Trace orqali request flow ko‘rildi
 ↓
Log correlation ID bilan qidirildi
 ↓
Root cause topildi
 ↓
Fix yoki rollback qilindi

4. Logs

Log — ilova ichida bo‘lgan voqealarni yozib borish.

Masalan:

User login qildi
Order yaratildi
Payment failed
Kafka event processed
DB timeout bo‘ldi

Java/Spring Boot’da ko‘p ishlatiladigan logging stack:

SLF4J API
 ↓
Logback yoki Log4j2 implementation

Ko‘pincha Spring Boot default holatda Logback bilan keladi.

5. Log level’lar

Log level’larni to‘g‘ri ishlatish muhim.

Misol:

log.info("Order created: orderId={}", orderId);

log.warn("Payment retry started: orderId={}, attempt={}", orderId, attempt);

log.error("Payment failed: orderId={}", orderId, exception);

6. System.out.println() nima uchun yomon?

Yomon:

System.out.println("Order created " + orderId);

Muammolari:

• log level yo‘q;

• format nazorat qilinmaydi;

• correlation ID yo‘q;

• JSON structured log qilish qiyin;

• production log collector bilan ishlashi noqulay;

• class/package bo‘yicha sozlash qiyin.

To‘g‘ri:

private static final Logger log = LoggerFactory.getLogger(OrderService.class);

log.info("Order created: orderId={}", orderId);

Yoki Lombok bilan:

@Slf4j
@Service
public class OrderService {

    public void createOrder(Long orderId) {
        log.info("Order created: orderId={}", orderId);
    }
}

7. Structured logging

Oddiy text log:

Order created successfully for user 15 with order 777

Bu odam uchun o‘qish oson, lekin log tizimlari uchun qidirish qiyinroq.

Structured logging — logni key-value yoki JSON formatda yozish.

Masalan:

{
  "level": "INFO",
  "message": "Order created",
  "orderId": 777,
  "userId": 15,
  "traceId": "abc123",
  "service": "order-service"
}

Afzalligi:

• orderId=777 bo‘yicha qidirish oson;

• Grafana Loki/ELK’da filter qilish oson;

• alert yozish oson;

• trace bilan bog‘lash oson.

8. Logback bilan JSON log

Spring Boot’da Logback ishlatilsa, JSON log uchun ko‘pincha logstash-logback-encoder ishlatiladi.

Dependency:

<dependency>
    <groupId>net.logstash.logback</groupId>
    <artifactId>logstash-logback-encoder</artifactId>
    <version>8.0</version>
</dependency>

logback-spring.xml:

<configuration>
    <appender name="JSON" class="ch.qos.logback.core.ConsoleAppender">
        <encoder class="net.logstash.logback.encoder.LogstashEncoder"/>
    </appender>

    <root level="INFO">
        <appender-ref ref="JSON"/>
    </root>
</configuration>

Container/Kubernetes’da log odatda stdout’ga chiqadi:

App → stdout → Fluent Bit/Logstash/Promtail → storage

9. MDC nima?

MDC — Mapped Diagnostic Context.

MDC loglarga avtomatik qo‘shimcha context qo‘shish uchun ishlatiladi.

Masalan:

traceId
requestId
userId
tenantId
correlationId

Agar bitta request davomida 10 ta class log yozsa, har bir logga correlationId avtomatik qo‘shiladi.

9.1. MDC filter misol

@Component
public class CorrelationIdFilter extends OncePerRequestFilter {

    private static final String HEADER = "X-Correlation-Id";

    @Override
    protected void doFilterInternal(
            HttpServletRequest request,
            HttpServletResponse response,
            FilterChain filterChain
    ) throws ServletException, IOException {

        String correlationId = request.getHeader(HEADER);

        if (correlationId == null || correlationId.isBlank()) {
            correlationId = UUID.randomUUID().toString();
        }

        try {
            MDC.put("correlationId", correlationId);
            response.setHeader(HEADER, correlationId);
            filterChain.doFilter(request, response);
        } finally {
            MDC.clear();
        }
    }
}

Muhim joy:

finally {
    MDC.clear();
}

Thread pool ishlatilganda MDC tozalanmasa, boshqa requestga eski context o‘tib ketishi mumkin.

10. Correlation ID

Correlation ID — bitta requestni bir nechta servis bo‘ylab kuzatish uchun ID.

Flow:

Client
 ↓ X-Correlation-Id: abc-123
Gateway
 ↓ X-Correlation-Id: abc-123
Order Service
 ↓ X-Correlation-Id: abc-123
Payment Service
 ↓ X-Correlation-Id: abc-123
Inventory Service

Hamma servis logida shu ID bo‘lsa, requestni topish oson:

[abc-123] Order created
[abc-123] Payment started
[abc-123] Payment timeout

11. Loglarda nima yozish kerak?

Yaxshi log:

log.info("Order created: orderId={}, userId={}", orderId, userId);

Yaxshi error log:

log.error("Payment failed: orderId={}, provider={}, reason={}",
        orderId, provider, reason, exception);

Yomon log:

log.info("Started");
log.info("Done");
log.error("Error");

Bunday loglar production’da foydasiz.

12. Loglarda nima yozmaslik kerak?

Sensitive data logga chiqmasligi kerak:

password
JWT token
refresh token
card number
CVV
private key
secret
API key
personal data

Yomon:

log.info("Login request: username={}, password={}", username, password);

Yomon:

log.info("Authorization header: {}", token);

To‘g‘ri:

log.info("Login attempt: username={}", username);

Token kerak bo‘lsa ham to‘liq emas, maskalab:

log.debug("Token received: tokenPrefix={}", token.substring(0, 10));

Lekin production’da tokenni umuman yozmaslik yaxshiroq.

13. Metrics

Metrics — raqamli o‘lchovlar.

Masalan:

request count
request latency
error rate
CPU usage
memory usage
GC pause
DB connection pool usage
Kafka consumer lag
queue depth
cache hit ratio

Metrics savollarga javob beradi:

Qancha request kelyapti?
Latency qancha?
Xatolar oshdimi?
Memory o‘sib boryaptimi?
DB pool to‘lib qoldimi?
Kafka consumer orqada qolyaptimi?

14. RED va USE metodlari

RED method

Microservice/API uchun juda foydali:

Masalan:

/orders endpoint:
Rate: 500 req/min
Errors: 2%
Duration p95: 350ms

USE method

Infrastructure/resource uchun:

Masalan DB pool:

Utilization: 95%
Saturation: pending threads 50
Errors: connection timeout

15. Micrometer

Micrometer — Spring Boot’da metrics uchun asosiy abstraction.

Spring Boot Actuator bilan birga ishlatiladi.

Dependency:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>

<dependency>
    <groupId>io.micrometer</groupId>
    <artifactId>micrometer-registry-prometheus</artifactId>
</dependency>

Config:

management:
  endpoints:
    web:
      exposure:
        include: health,info,metrics,prometheus

  metrics:
    tags:
      application: order-service

Prometheus endpoint:

/actuator/prometheus

16. Custom metrics

Ba’zida default metric yetmaydi. Biznes metric kerak bo‘ladi.

Masalan:

order.created.count
payment.failed.count
checkout.duration

Counter

Counter faqat oshib boradi.

@Service
public class OrderMetrics {

    private final Counter orderCreatedCounter;

    public OrderMetrics(MeterRegistry meterRegistry) {
        this.orderCreatedCounter = Counter.builder("order_created_total")
                .description("Total created orders")
                .tag("service", "order-service")
                .register(meterRegistry);
    }

    public void incrementOrderCreated() {
        orderCreatedCounter.increment();
    }
}

Timer

Timer method yoki operation vaqtini o‘lchaydi.

@Service
@RequiredArgsConstructor
public class PaymentService {

    private final MeterRegistry meterRegistry;

    public PaymentResult pay(PaymentRequest request) {
        return Timer.builder("payment_duration")
                .tag("provider", request.provider())
                .register(meterRegistry)
                .record(() -> callPaymentProvider(request));
    }

    private PaymentResult callPaymentProvider(PaymentRequest request) {
        // external payment call
        return new PaymentResult("SUCCESS");
    }
}

17. Metric cardinality

Metrics’da eng xavfli xatolardan biri — high cardinality.

Yomon:

Counter.builder("request_count")
        .tag("userId", userId.toString())
        .register(meterRegistry);

Agar 1 million user bo‘lsa, 1 million time series paydo bo‘ladi.

Bu Prometheus/Grafana’ni ezib qo‘yadi.

Yaxshi taglar:

service
endpoint
method
status
exception
provider
region

Yomon taglar:

userId
orderId
email
phone
requestId
traceId

Qoida:

Metric tag ichiga unbounded qiymat qo‘shmang.

18. Prometheus

Prometheus — metrics yig‘ish va query qilish tizimi.

Prometheus pull model ishlatadi:

Prometheus → /actuator/prometheus → Spring Boot app

Ya’ni app Prometheus’ga yubormaydi. Prometheus app’dan o‘zi scrape qiladi.

Prometheus config misol:

scrape_configs:
  - job_name: 'order-service'
    metrics_path: '/actuator/prometheus'
    static_configs:
      - targets: ['order-service:8080']

Kubernetes’da ServiceMonitor/PodMonitor ishlatilishi mumkin.

19. PromQL qisqa kirish

Prometheus query tili — PromQL.

Request rate:

rate(http_server_requests_seconds_count[5m])

Error rate:

sum(rate(http_server_requests_seconds_count{status=~"5.."}[5m]))
/
sum(rate(http_server_requests_seconds_count[5m]))

p95 latency:

histogram_quantile(
  0.95,
  sum(rate(http_server_requests_seconds_bucket[5m])) by (le, uri)
)

CPU usage:

process_cpu_usage

JVM memory:

jvm_memory_used_bytes

GC pause:

jvm_gc_pause_seconds

20. Grafana dashboards

Grafana — metrics, logs va traces’ni vizual ko‘rsatish uchun dashboard tool.

Spring Boot service uchun minimal dashboard:

Request rate
Error rate
p50/p95/p99 latency
CPU usage
Memory usage
GC pause
Thread count
DB connection pool usage
Kafka consumer lag
HTTP client latency

Yaxshi dashboard savolga javob berishi kerak:

Servis tirikmi?
Traffic normalmi?
Xatolar oshdimi?
Latency yomonlashdimi?
Bottleneck qayerda?

21. Golden signals

Google SRE’da mashhur bo‘lgan 4 ta signal:

API dashboard’da shular bo‘lishi shart.

22. Distributed tracing

Distributed tracing — bitta requestning bir nechta servis bo‘ylab yo‘lini ko‘rsatadi.

Masalan:

POST /checkout
 ↓
Order Service: 80ms
 ↓
Payment Service: 1200ms
 ↓
Inventory Service: 50ms
 ↓
Notification Service: async

Trace ko‘rsatadi:

Qaysi servis sekin?
Qaysi external call timeout?
Qaysi DB query uzoq?
Qaysi span error berdi?

23. Trace, span, traceId

Trace

Bitta requestning to‘liq yo‘li.

POST /checkout

Span

Trace ichidagi bitta operation.

OrderService.createOrder
PaymentClient.charge
InventoryClient.reserve
SQL SELECT orders

Trace ID

Butun request uchun bitta ID.

Span ID

Har bir operation uchun alohida ID.

Ko‘rinish:

traceId=abc123
 ├── spanId=1 gateway
 ├── spanId=2 order-service
 ├── spanId=3 payment-service
 └── spanId=4 inventory-service

24. OpenTelemetry

OpenTelemetry — logs, metrics, traces uchun vendor-neutral standart.

Ya’ni siz tracingni faqat Zipkin yoki Jaeger’ga bog‘lab qo‘ymaysiz.

Architecture:

Spring Boot App
 ↓
OpenTelemetry SDK/Agent
 ↓
OpenTelemetry Collector
 ↓
Jaeger/Zipkin/Tempo/Datadog/New Relic

Afzallik:

• vendor lock-in kamayadi;

• ko‘p til va frameworkni qo‘llaydi;

• microservice uchun standartlashgan;

• trace context servislar orasida uzatiladi.

25. OpenTelemetry Java agent

Eng oson yo‘l — Java agent ulash.

java \
  -javaagent:opentelemetry-javaagent.jar \
  -Dotel.service.name=order-service \
  -Dotel.exporter.otlp.endpoint=http://otel-collector:4318 \
  -jar app.jar

Docker/Kubernetes’da JAVA_TOOL_OPTIONS bilan berish mumkin:

env:
  - name: JAVA_TOOL_OPTIONS
    value: "-javaagent:/otel/opentelemetry-javaagent.jar"
  - name: OTEL_SERVICE_NAME
    value: "order-service"
  - name: OTEL_EXPORTER_OTLP_ENDPOINT
    value: "http://otel-collector:4318"

Agent ko‘p narsani avtomatik instrument qiladi:

Spring MVC/WebFlux
RestTemplate/WebClient
JDBC
Kafka
Redis
HTTP clients

26. Zipkin

Zipkin — distributed tracing backend.

Spring Boot’dan trace yuboriladi, Zipkin UI’da request flow ko‘rinadi.

Flow:

App → traces → Zipkin → UI

Zipkin’da ko‘rish mumkin:

• trace duration;

• servislar orasidagi flow;

• qaysi span sekin;

• error span;

• dependency graph.

27. Trace propagation

Tracing ishlashi uchun trace context servisdan servisga o‘tishi kerak.

HTTP’da headerlar orqali:

traceparent
tracestate

Yoki eski formatlarda:

X-B3-TraceId
X-B3-SpanId

Masalan:

Gateway trace yaratadi
 ↓
Order Service shu trace bilan davom etadi
 ↓
Payment Service shu trace bilan davom etadi

Agar header uzatilmasa, har servis alohida trace yaratadi. Flow uziladi.

28. Logs + traces bog‘lash

Eng yaxshi holat:

Log ichida traceId bor
Trace ichida error span bor
Dashboarddan tracega o‘tasiz
Tracedan loglarga o‘tasiz

Log misol:

{
  "level": "ERROR",
  "message": "Payment provider timeout",
  "traceId": "abc123",
  "spanId": "def456",
  "orderId": 777
}

Shunda:

traceId=abc123

bo‘yicha loglarni topish mumkin.

29. Alerting strategy

Alerting — muammo bo‘lganda xabar berish.

Lekin yomon alerting developerlarni charchatadi.

Yomon alert:

CPU 80% bo‘ldi

Bu har doim incident emas.

Yaxshi alert:

p95 latency 5 daqiqa davomida 1 sekunddan yuqori
va error rate 5% dan yuqori

Yoki:

Payment failed rate 10 daqiqa davomida 3x oshdi

30. Alert turlari

Symptom-based alert

Userga ta’sir qilayotgan holat bo‘yicha.

Masalan:

Error rate oshdi
Latency oshdi
Checkout success rate tushdi

Bu eng muhim.

Cause-based alert

Ichki sabab bo‘yicha.

Masalan:

CPU high
Memory high
DB pool exhausted
Kafka lag high

Bu foydali, lekin user impact bilan bog‘lash kerak.

31. SLO, SLI, SLA

SLI — Service Level Indicator

O‘lchanadigan signal.

Masalan:

99% requests < 300ms
Error rate < 0.1%
Availability 99.9%

SLO — Service Level Objective

Ichki maqsad.

Masalan:

Checkout API p95 latency < 500ms
Monthly availability > 99.9%

SLA — Service Level Agreement

Mijoz bilan kelishilgan majburiyat.

Masalan:

Agar availability 99.5% dan past bo‘lsa, kompensatsiya beriladi.

Senior uchun muhim farq:

SLI — o‘lchov
SLO — ichki target
SLA — tashqi shartnoma

32. Error budget

Agar SLO:

99.9% availability

bo‘lsa, demak 0.1% xatoga ruxsat bor. Shu — error budget.

Error budget tugasa:

Yangi feature chiqarish sekinlashadi
Reliability muammolari tuzatiladi

Bu engineering va biznes orasida balans beradi.

33. Real incident tahlil misoli

Scenario:

POST /checkout p95 latency 300ms dan 4s ga chiqdi

1-qadam: Dashboard

Grafana ko‘rsatdi:

checkout latency high
error rate 2%
CPU normal
memory normal

2-qadam: Trace

Trace ochildi:

Order Service: 80ms
Payment Service: 3800ms
Inventory Service: 40ms

Demak sekinlik Payment tomonda.

3-qadam: Payment logs

traceId bo‘yicha log:

Payment provider timeout
Retry attempt=3

4-qadam: Metrics

Payment client metric:

external_payment_latency p95 = 3500ms
retry_count oshgan

5-qadam: Root cause

External payment provider sekinlashgan. Retry ko‘p urinyapti va latency oshiryapti.

6-qadam: Fix

• timeout kamaytirildi;

• retry count pasaytirildi;

• circuit breaker sozlandi;

• fallback holat qo‘shildi;

• alert threshold qayta ko‘rildi.

34. Observability production architecture

Oddiy cloud-native setup:

Spring Boot App
 ├── logs → stdout → Fluent Bit/Promtail → Loki/Elasticsearch
 ├── metrics → /actuator/prometheus → Prometheus → Grafana
 └── traces → OpenTelemetry → Tempo/Jaeger/Zipkin

Kubernetes’da:

Pod logs
Pod metrics
Service metrics
Node metrics
Ingress metrics
Application metrics
Trace context

hammasi birga ishlaydi.

35. Spring Boot Actuator

Actuator production observability uchun juda foydali.

Endpointlar:

/actuator/health
/actuator/info
/actuator/metrics
/actuator/prometheus
/actuator/loggers

Config:

management:
  endpoints:
    web:
      exposure:
        include: health,info,metrics,prometheus

  endpoint:
    health:
      probes:
        enabled: true
      show-details: never

Security muhim:

/actuator/env, /actuator/heapdump, /actuator/threaddump kabi endpointlarni productionda ochiq qoldirmang.

36. Health vs Observability

Health check — faqat servis trafik qabul qila oladimi yoki yo‘qligini bildiradi.

Observability esa sababni topadi.

Yomon yondashuv:

Health UP → demak hammasi yaxshi

Bu noto‘g‘ri.

Servis UP bo‘lishi mumkin, lekin:

latency 5 sekund
error rate 10%
Kafka lag 1 million
DB pool 100% busy

bo‘lishi mumkin.

37. Kafka observability

Kafka ishlatilsa, alohida kuzatish kerak:

consumer lag
message processing latency
failed messages
retry count
DLT count
rebalance count
producer send error
topic throughput

Consumer lag muhim:

lag = brokerdagi oxirgi offset - consumer o‘qigan offset

Agar lag oshaversa:

• consumer sekin;

• DB sekin;

• external API sekin;

• partition kam;

• consumer instance kam;

• message processing ichida blocking bor.

38. Database observability

Java app sekinlashsa, ko‘pincha DB sabab bo‘ladi.

Kuzatish kerak:

query latency
slow queries
connection pool active/idle
connection timeout
deadlock
lock wait
index usage
transaction duration

HikariCP metrics:

hikaricp_connections_active
hikaricp_connections_idle
hikaricp_connections_pending
hikaricp_connections_timeout_total

Agar pending oshsa, threadlar connection kutyapti.

39. JVM observability

Java uchun quyidagilar muhim:

heap used
non-heap used
GC pause
GC frequency
thread count
blocked threads
class count
CPU usage
allocation rate

GC pause oshsa:

• heap kichik;

• allocation ko‘p;

• memory leak;

• noto‘g‘ri GC;

• katta objectlar ko‘p;

bo‘lishi mumkin.

40. Senior xatolari

Xato 1: Faqat logga ishonish

Log muhim, lekin metric va trace bo‘lmasa, umumiy holat ko‘rinmaydi.

Xato 2: Logga sensitive data yozish

Token, password, karta, secret logga chiqmasligi kerak.

Xato 3: Correlation ID yo‘q

Microservice’da requestni kuzatib bo‘lmaydi.

Xato 4: Metric tagga userId/orderId qo‘shish

Prometheus portlashi mumkin. High cardinality.

Xato 5: Dashboard bor, alert yo‘q

Muammo bo‘ladi, lekin hech kim bilmaydi.

Xato 6: Alert juda ko‘p

Har kichik signalga alert bo‘lsa, odamlar alertni e’tiborsiz qoldiradi.

Xato 7: Actuator endpointlarni ochiq qoldirish

Security incident bo‘lishi mumkin.

Xato 8: Trace samplingni o‘ylamaslik

Har requestni trace qilish katta trafficda qimmat bo‘lishi mumkin.

41. Senior checklist

Production Java service uchun observability checklist:

Structured JSON log bormi?
Loglarda service name, traceId, correlationId bormi?
Sensitive data logga chiqmayaptimi?
Actuator yoqilganmi?
Prometheus metrics chiqyaptimi?
Custom business metrics bormi?
p95/p99 latency kuzatilyaptimi?
Error rate kuzatilyaptimi?
DB pool metrics bormi?
Kafka lag metrics bormi?
JVM heap/GC/thread metrics bormi?
Distributed tracing ulanganmi?
TraceId log bilan bog‘langanmi?
Grafana dashboard bormi?
Alertlar user impact asosidami?
Runbook bormi?

42. Amaliy topshiriqlar

Topshiriq 1: Structured logging

Spring Boot app’da JSON log yoqing.

Talab:

level
message
service
correlationId
traceId
timestamp

logda chiqsin.

Topshiriq 2: Correlation ID

Filter yozing:

X-Correlation-Id bor bo‘lsa oling
Yo‘q bo‘lsa generate qiling
Response headerga qaytaring
MDCga yozing
finally’da tozalang

Topshiriq 3: Micrometer + Prometheus

Actuator va Prometheus endpoint yoqing:

/actuator/prometheus

Keyin custom counter qo‘shing:

order_created_total
payment_failed_total

Topshiriq 4: Grafana dashboard

Quyidagi panellarni yarating:

Request rate
Error rate
p95 latency
JVM heap
GC pause
DB active connections
Kafka consumer lag

Topshiriq 5: Distributed tracing

OpenTelemetry Java agent ulang.

Talab:

order-service → payment-service

HTTP call trace’da bitta flow sifatida ko‘rinsin.

43. Interview savollari

Logs

• Structured logging nima?

• MDC nima uchun kerak?

• Correlation ID nima?

• Loglarda sensitive data masalasi qanday hal qilinadi?

Metrics

• Metrics nima?

• Counter, gauge, timer farqi?

• Prometheus pull model nima?

• High cardinality nima?

• RED method nima?

Tracing

• Trace va span farqi?

• OpenTelemetry nima?

• Trace propagation nima?

• Zipkin/Jaeger nima qiladi?

Alerting

• Alert threshold qanday tanlanadi?

• SLI/SLO/SLA farqi?

• Error budget nima?

• Symptom-based alert nima uchun yaxshi?

44. Qisqa xulosa

Observability Senior Java developer uchun production’da omon qolish skillidir.

Asosiy fikrlar:

Logs — nima bo‘ldi?
Metrics — qanchalik ko‘p va qanchalik yomon?
Traces — request qayerdan o‘tdi va qayerda sekinlashdi?

Senior developer:

• structured logging qiladi;

• MDC/correlation ID ishlatadi;

• Micrometer bilan metrics chiqaradi;

• Prometheus/Grafana bilan kuzatadi;

• OpenTelemetry bilan tracing qiladi;

• alertlarni user impact asosida sozlaydi;

• sensitive data logga chiqmasligini nazorat qiladi.

Eng muhim qoida:

Production’da muammo bo‘lganda “nima bo‘ldi ekan?” deb taxmin qilinmaydi. Observability shunday quriladiki, muammoning sababini metrika, log va trace orqali aniq ko‘rish mumkin.