Ma’lumotlar arxitekturasi (Data Architecture)

Endi arxitektor uchun eng muhim qatlamlardan biri:

Data architecture - systemda ma’lumot qanday saqlanadi, qanday oqadi, qanday himoyalanadi, qanday o‘zgaradi va vaqt o‘tib qanday boshqariladi.

Bu mavzu ichida quyidagilar bor: Polyglot persistence strategy, Data mesh concepts, Event streaming as system of record, Schema evolution & compatibility, GDPR & data privacy patterns.

1. Data Architecture nima?

Oddiy developer ko‘pincha shunday o‘ylaydi:

Entity yozamiz
Repository yozamiz
Databasega save qilamiz

Arxitektor esa boshqa savollarni beradi:

Qaysi data qayerda saqlanadi?
Qaysi service data egasi?
Data kim tomonidan o‘zgartiriladi?
Read va write modeli bir xilmi?
Data qancha vaqt saqlanadi?
Backup va restore qanday?
Schema o‘zgarsa eski systemlar buzilmaydimi?
Sensitive data qanday himoyalanadi?
Report va analytics operational DB’ni qiynamaydimi?

Ya’ni data architecture - faqat database tanlash emas.

Bu butun system bo‘ylab data ownership, consistency, flow, privacy, lifecycle masalasi.

2. Data Architecture nega muhim?

Kichik loyihada bitta PostgreSQL yetadi:

Spring Boot App → PostgreSQL

Lekin loyiha kattalashganda muammolar chiqadi:

Report query production DB’ni sekinlashtiradi
Bir service boshqa service table’ini to‘g‘ridan-to‘g‘ri o‘qiydi
Schema o‘zgarsa boshqa app buziladi
Customer data hamma joyda duplicate bo‘lib ketadi
Sensitive data loglarga chiqib ketadi
Delete qilingan user analytics’da qoladi
Event format o‘zgarsa consumerlar yiqiladi

Arxitektor shu muammolarni oldindan kamaytiradi.

3. Data Ownership

Data ownership nima?

Har bir muhim data uchun aniq egasi bo‘lishi kerak.

Masalan e-commerce system:

Asosiy qoida:

Bir data’ning faqat bitta owner’i bo‘lishi kerak.

Yomon holat

Order Service → user_db.users table’ini to‘g‘ridan-to‘g‘ri o‘qiydi
Payment Service → order_db.orders table’ini update qiladi
Report Service → hamma DB table’lariga JOIN qiladi

Bu boshida tez ishlaydi. Keyin chaos boshlanadi.

Muammo:

Qaysi service data egasi ekani noaniq
Schema o‘zgarsa boshqa servicelar buziladi
Data consistency buziladi
Teamlar bir-birini bloklaydi

Yaxshi holat

Order Service user haqida kerakli ma’lumotni Auth/User API orqali oladi
Payment Service order statusni event orqali biladi
Report Service read model yoki data warehouse’dan o‘qiydi

Ya’ni service boshqa service’ning ichki database’iga kirmaydi.

4. Database per Service

Microservices’da ko‘p ishlatiladigan qoida:

Har service o‘z database’iga ega bo‘ladi.

User Service       → user_db
Order Service      → order_db
Payment Service    → payment_db
Notification       → notification_db

Bu shuni anglatadi:

Order Service payment_db ichiga kirmaydi
Payment Service order_db ichiga kirmaydi

Ular API yoki event orqali gaplashadi.

Afzalliklari

Kamchiliklari

Arxitektor xulosasi

Database per service - microservices uchun kuchli yondashuv, lekin u bepul emas. Kichik teamda bu complexity juda qimmatga tushishi mumkin.

Modular monolith’da esa bitta DB bo‘lishi mumkin, lekin table ownership baribir aniq bo‘lishi kerak.

5. Polyglot Persistence

Polyglot persistence nima?

Polyglot persistence - har xil data turi uchun har xil database ishlatish.

Ya’ni:

Hamma narsa uchun bitta PostgreSQL ishlatish shart emas.

Masalan:

Real misol: Sales Agent system

PostgreSQL:
- agents
- customers
- orders
- visits

Redis:
- cache
- session
- rate limit

Kafka/RabbitMQ:
- order_created event
- notification jobs

OpenSearch:
- customer/product search

S3/MinIO:
- uploaded images
- reports
- invoices

ClickHouse:
- analytics/reporting

Bu polyglot persistence.

Afzalliklari

Kamchiliklari

Eng katta xato

Yomon yondashuv:

PostgreSQL ham bor
MongoDB ham bor
Redis ham bor
Elasticsearch ham bor
Kafka ham bor
Lekin aniq sabab yo‘q

Bu architecture emas, bu texnologiya yig‘indisi.

Yaxshi yondashuv:

PostgreSQL - transactional source of truth
Redis - temporary cache
OpenSearch - full-text search projection
Kafka - event transport
ClickHouse - analytics

Har birining roli aniq.

6. Source of Truth

Source of truth nima?

Source of truth - ma’lumotning eng ishonchli, rasmiy manbasi.

Masalan:

Order status uchun source of truth → Order Service DB
Payment status uchun source of truth → Payment Service DB
Customer profile uchun source of truth → Customer Service DB

Agar Redis cache’da boshqa qiymat, DB’da boshqa qiymat bo‘lsa, qaysi biri haqiqiy?

Javob oldindan aniq bo‘lishi kerak.

Misol

PostgreSQL:
order_id=1001, status=PAID

Redis:
order_id=1001, status=PENDING

Bu holatda PostgreSQL source of truth bo‘lsa:

Redis noto‘g‘ri yoki eski cache hisoblanadi.

Qoida

Cache source of truth bo‘lmasligi kerak, agar maxsus shunday design qilinmagan bo‘lsa.

Ko‘p systemlarda:

Database = source of truth
Cache = tezlashtirish uchun nusxa
Search index = query uchun projection
Analytics DB = report uchun nusxa

7. Event Streaming as System of Record

Bu nima degani?

Odatda system of record - database.

Masalan:

orders table = order holatining asosiy manbasi

Lekin event-driven architecture’da ba’zan event log asosiy manba bo‘lishi mumkin:

OrderCreated
OrderPaid
OrderShipped
OrderCancelled

Ya’ni system holati eventlardan tiklanadi.

Bu Event Sourcing’ga yaqin.

Kafka misol

Kafka topic: order-events

1. OrderCreated
2. OrderItemAdded
3. PaymentReceived
4. OrderConfirmed
5. OrderShipped

Read model, analytics, notification - hammasi shu eventlardan qurilishi mumkin.

Kafka events → Order Projection DB
Kafka events → Analytics DB
Kafka events → Notification Service
Kafka events → Search Index

Afzalliklari

Kamchiliklari

Qachon mos?

Event streaming as system of record mos:

• audit juda muhim;

• history/replay kerak;

• ko‘p system bir xil eventlardan foydalanadi;

• real-time analytics kerak;

• biznes jarayon eventlar ketma-ketligi bilan ifodalanadi.

Mos emas:

• oddiy CRUD;

• kichik admin panel;

• team event-driven tajribasiz;

• event schema boshqarish jarayoni yo‘q.

8. Schema Evolution

Schema evolution nima?

System ishlayotgan paytda data modeli o‘zgaradi.

Masalan:

customer table’ga phone_number qo‘shildi
order event’ga deliveryType qo‘shildi
API response’dan field olib tashlandi
JSON structure o‘zgardi

Bu o‘zgarishlar eski client yoki consumerlarni buzmasligi kerak.

Database schema evolution

Yomon migration:

ALTER TABLE customers ADD COLUMN phone_number VARCHAR(20) NOT NULL;

Agar table’da eski rowlar bo‘lsa, bu production’da muammo berishi mumkin.

Yaxshiroq tartib:

1. Nullable column qo‘shish
2. App yangi fieldni yozishni boshlaydi
3. Eski data backfill qilinadi
4. Validation qo‘shiladi
5. Keyin NOT NULL constraint qo‘yiladi

Safe migration pattern

Masalan full_name o‘rniga first_name, last_name qilmoqchimiz.

Yomon:

full_name column delete
first_name qo‘shish
last_name qo‘shish
app deploy

Bu downtime yoki bug keltiradi.

Yaxshi:

1. first_name, last_name nullable qo‘shiladi
2. app ikkala formatni ham o‘qiy oladi
3. yangi yozuvlar first_name/last_name yozadi
4. eski data backfill qilinadi
5. hamma consumer yangilanadi
6. full_name deprecated bo‘ladi
7. keyin delete qilinadi

Bu expand-contract pattern.

9. Event Schema Compatibility

Event-driven systemda schema evolution yanada muhim.

Masalan eski event:

{
  "eventType": "OrderCreated",
  "orderId": "1001",
  "total": 250000
}

Yangi event:

{
  "eventType": "OrderCreated",
  "orderId": "1001",
  "total": 250000,
  "currency": "UZS"
}

Bu odatda backward compatible, chunki yangi field qo‘shildi.

Lekin bu xavfli:

{
  "eventType": "OrderCreated",
  "id": "1001",
  "amount": 250000
}

Bu eski consumerlarni buzadi, chunki orderId va total yo‘q.

Compatibility turlari

Event schema qoidalari

Yaxshi qoidalar:

Field qo‘shish mumkin, lekin optional/default bilan
Field nomini o‘zgartirmang
Fieldni darrov delete qilmang
Meaning o‘zgartirmang
Enum value qo‘shishda ehtiyot bo‘ling
Event version yuriting
Schema registry ishlating
Consumerlarni tolerant qiling

Versioning misol

{
  "eventType": "OrderCreated",
  "eventVersion": 2,
  "orderId": "1001",
  "total": 250000,
  "currency": "UZS"
}

Yoki topic bo‘yicha:

order-created-v1
order-created-v2

Lekin topicni versionlash ham ko‘payib ketsa boshqarish qiyinlashadi.

10. Data Mesh Concepts

Data mesh nima?

Data mesh - data ownership’ni markaziy data teamdan domain teamlarga tarqatish yondashuvi.

Oddiy model:

Hamma data → Central Data Team → Data Warehouse

Muammo:

Central team bottleneck bo‘ladi
Domain ma’nosini yaxshi bilmasligi mumkin
Data sifati pasayadi
Reportlar kechikadi

Data mesh’da esa:

Sales domain → o‘z data product’iga javobgar
Orders domain → o‘z data product’iga javobgar
Payments domain → o‘z data product’iga javobgar

Data product nima?

Data product - boshqa teamlar ishlata oladigan, hujjatlashtirilgan, sifatli data manbasi.

Masalan:

orders_daily_summary
customer_activity_events
payment_success_metrics
agent_visit_history

Data product’da bo‘lishi kerak:

Owner
Schema
Description
Quality rules
SLA/SLO
Access policy
Versioning
Lineage

Data mesh prinsiplari

Qachon kerak?

Data mesh kerak bo‘lishi mumkin:

• kompaniya katta bo‘lsa;

• domain teamlar ko‘p bo‘lsa;

• central data team bottleneck bo‘lsa;

• analytics va reporting ko‘p bo‘lsa;

• data ownership chalkash bo‘lsa.

Kichik projectda data mesh ortiqcha bo‘lishi mumkin.

11. Operational DB vs Analytical DB

Farqi

Yomon holat:

Production PostgreSQL’da og‘ir report query ishlaydi
JOIN + GROUP BY + millions rows
Natija: API sekinlashadi

Yaxshi holat:

Production DB → CDC/Event/Pipeline → Analytics DB

Masalan:

PostgreSQL → Kafka/Debezium → ClickHouse

Yoki:

PostgreSQL → scheduled ETL → BigQuery

Sales Agent misoli

Operational DB:

orders
visits
customers
agents

Analytical DB:

daily_agent_sales
monthly_region_performance
customer_visit_frequency
product_sales_trends

Supervisor dashboard analytics DB’dan o‘qisa, production transaction DB kamroq qiynaladi.

12. CDC - Change Data Capture

CDC nima?

CDC - database’dagi o‘zgarishlarni ushlab, boshqa systemlarga yuborish.

Masalan:

orders table’da yangi row paydo bo‘ldi
CDC uni eventga aylantirdi
Kafka’ga yubordi
Analytics DB uni qabul qildi

PostgreSQL WAL → Debezium → Kafka → ClickHouse/OpenSearch

Qachon foydali?

CDC foydali:

• legacy systemdan event olish kerak bo‘lsa;

• analytics pipeline kerak bo‘lsa;

• search index yangilanishi kerak bo‘lsa;

• DB o‘zgarishlarini boshqa servicelarga tarqatish kerak bo‘lsa.

Ehtiyot bo‘lish kerak

CDC eventlari ko‘pincha technical bo‘ladi:

row inserted
row updated
column changed

Business event esa boshqa:

OrderConfirmed
PaymentReceived
CustomerBlocked

Arxitektor farqni bilishi kerak.

CDC integration uchun yaxshi, lekin business eventlarning to‘liq o‘rnini har doim bosmaydi.

13. Data Privacy

Data privacy nima?

Data privacy - user yoki biznesga tegishli sensitive data’ni noto‘g‘ri ishlatmaslik, oshkor qilmaslik va kerak bo‘lganda o‘chirish.

Sensitive data:

phone number
passport data
location
payment info
address
personal messages
health data
financial data

Privacy by Design

Arxitektor boshidan privacy’ni o‘ylashi kerak.

Savollar:

Bu data bizga haqiqatan kerakmi?
Qancha vaqt saqlaymiz?
Kim ko‘ra oladi?
Logga chiqib ketmaydimi?
Backup ichida qancha turadi?
User delete so‘rasa, qayerlardan o‘chiramiz?
Analytics’da anonymized bo‘ladimi?

14. GDPR va data privacy patterns

GDPR Yevropa regulation’i bo‘lsa ham, uning prinsiplari umumiy foydali.

Muhim patternlar

Data minimization misol

Yomon:

Registration:
- ism
- telefon
- passport
- manzil
- tug‘ilgan sana
- karta raqami

Agar appga faqat login kerak bo‘lsa, bu ortiqcha.

Yaxshi:

Registration:
- phone number
- password/OTP

Kerak bo‘lsa keyin qo‘shimcha data olinadi.

Pseudonymization

Sensitive identity alohida saqlanadi.

orders table:
user_id = 123
total = 250000

user_private_data table:
user_id = 123
phone = +998...
passport = ...

Analytics’da faqat user_id yoki hashed identifier ishlatiladi.

Anonymization

Shaxsni qayta aniqlab bo‘lmaydigan qilish.

Masalan analytics uchun:

User Ali, phone +99890... → Region=Tashkent, AgeGroup=25-34

Endi individual user emas, aggregate data ishlatiladi.

15. Encryption

Encryption at rest

Data diskda shifrlangan bo‘ladi.

Masalan:

Database disk encryption
S3 bucket encryption
Backup encryption

Encryption in transit

Data tarmoq orqali ketayotganda shifrlanadi.

HTTPS/TLS
mTLS service-to-service
TLS for database connection

Field-level encryption

Ayrim fieldlar application darajasida shifrlanadi.

Masalan:

passport_number
card_token
medical_info

DB admin ham plain text ko‘rmasligi mumkin.

Hashing vs Encryption

Password uchun encryption emas, hashing kerak:

BCrypt
Argon2
PBKDF2

16. Data Retention

Retention nima?

Retention - data qancha vaqt saqlanishini belgilash.

Masalan:

Login audit logs → 1 yil
Payment records → 5 yil
Temporary OTP → 5 daqiqa
Raw location data → 30 kun
Aggregated analytics → 3 yil

Agar retention bo‘lmasa, data cheksiz ko‘payadi va risk oshadi.

Retention policy’da bo‘lishi kerak

Data turi
Saqlash muddati
O‘chirish jarayoni
Archive kerakmi?
Legal talab bormi?
Backup’dan qachon yo‘qoladi?
Kim javobgar?

17. Backup va Restore

Backup borligi yetarli emas.

Restore test qilinmagan backup - ishonchli backup emas.

Backup strategy

RPO/RTO

RPO = Recovery Point Objective
Qancha data yo‘qotish mumkin?

RTO = Recovery Time Objective
Qancha vaqtda systemni tiklash kerak?

Misol:

RPO = 5 minutes
Demak oxirgi 5 daqiqalik data yo‘qolishi mumkin.

RTO = 30 minutes
Demak system 30 daqiqa ichida tiklanishi kerak.

18. Data Quality

Data quality nima?

Data borligi yetarli emas. U to‘g‘ri, toza va ishonchli bo‘lishi kerak.

Data quality muammolari:

duplicate customer
phone number noto‘g‘ri formatda
order status impossible holatda
payment bor, lekin order topilmaydi
created_at null
negative price

Data quality checks

NOT NULL constraints
Foreign key constraints
Unique constraints
Validation rules
Deduplication jobs
Data reconciliation
Monitoring dashboards
Anomaly detection

Misol

Order status transition:

CREATED → PAID → SHIPPED → DELIVERED

Noto‘g‘ri:

CREATED → DELIVERED
PAID → CREATED
CANCELLED → SHIPPED

Bunday qoidalar domain layer va database constraintlarda himoyalanishi kerak.

19. Master Data Management

MDM nima?

Master Data Management - kompaniyadagi asosiy data obyektlarini yagona va to‘g‘ri boshqarish.

Masalan:

Customer
Product
Region
Branch
Employee
Supplier

Muammo:

Sales systemda customer nomi boshqacha
Billing systemda boshqacha
Support systemda boshqacha

MDM shu chalkashlikni kamaytiradi.

Golden Record

Golden record - bitta entity uchun eng ishonchli birlashtirilgan record.

Masalan bitta customer 3 joyda bor:

CRM: Ali Valiyev, +99890111...
Billing: A. Valiyev, +99890111...
Support: Ali, +99890111...

MDM ularni bitta customer sifatida bog‘laydi.

20. Data Architecture qarorlarini qanday tanlash kerak?

Arxitektor quyidagi tartibda o‘ylaydi:

1. Data domainlarini aniqlash
2. Har domain uchun owner belgilash
3. Source of truthni aniqlash
4. Consistency talabini aniqlash
5. Read/write patternni tahlil qilish
6. Storage turini tanlash
7. Schema evolution strategiyasini belgilash
8. Privacy va retention qoidalarini qo‘shish
9. Backup/restore strategiyasini belgilash
10. Monitoring va data quality checks qo‘shish

21. Real loyiha uchun data architecture: Sales Agent / Supervisor

Sizdagi ikki Android app va backend uchun amaliy variant:

Android Apps
  - Supervisor
  - Sales Agent
        |
API Gateway / Nginx
        |
Spring Boot Modular Monolith
        |
PostgreSQL
        |
Redis
        |
RabbitMQ/Kafka
        |
Analytics/Search/Storage

Boshlang‘ich data architecture

PostgreSQL = source of truth

Modules:
- auth
- agents
- customers
- visits
- orders
- reports
- notifications

Table ownership:

Qo‘shimcha storage

Eventlar

CustomerCreated
VisitStarted
VisitCompleted
OrderCreated
OrderConfirmed
PaymentReceived
NotificationRequested

Bu eventlar:

notification
reporting
audit
analytics
search indexing

uchun ishlatilishi mumkin.

22. Data Architecture bo‘yicha katta xatolar

1. Hamma data’ni bitta "shared database"ga tashlash

Boshida oson. Keyin owner yo‘qoladi.

2. Cache’ni source of truth qilib yuborish

Redis o‘chsa yoki eski qiymat saqlasa, system noto‘g‘ri ishlaydi.

3. Schema migration’ni production’da birdan qilish

Safe migration pattern bo‘lmasa, downtime bo‘ladi.

4. Event schema’ni o‘ylamasdan o‘zgartirish

Consumerlar yiqiladi.

5. Sensitive data’ni logga chiqarish

Masalan:

password
token
passport
card number
phone verification code

logga chiqmasligi kerak.

6. Backup bor, lekin restore test yo‘q

Bu eng xavfli holatlardan biri.

7. Analytics query production DB’ni o‘ldirishi

Reportlar alohida read model yoki analytical DB’dan o‘qishi kerak.

23. Arxitektor uchun qisqa checklist

Data owner aniqmi?
Source of truth aniqmi?
Service boshqa servicening DB’siga kirmaydimi?
Schema migration safe usuldami?
Event schema versionlanganmi?
Sensitive data logga chiqmayaptimi?
Retention policy bormi?
Backup va restore test qilinganmi?
Read-heavy querylar alohida modelga chiqarilganmi?
Cache invalidation strategiyasi bormi?
Data quality checks bormi?

24. Yakuniy formula

Data Architecture = Ownership
+ Source of Truth
+ Storage Strategy
+ Schema Evolution
+ Data Flow
+ Privacy
+ Backup/Restore
+ Quality

Arxitektor darajasidagi asosiy fikr:

Data systemning eng qimmat aktivi. Kodni qayta yozish mumkin, lekin noto‘g‘ri yo‘qotilgan yoki buzilgan data’ni tiklash juda qiyin.

Java/Spring Boot loyihalar uchun amaliy start:

PostgreSQL as source of truth
+ clear module/table ownership
+ Redis only for cache/session
+ safe migrations with Flyway/Liquibase
+ events for integration
+ read replica/read model for reports
+ S3/MinIO for files
+ encryption + retention policy
+ backup restore testing