@startuml skinparam sequenceMessageAlign center skinparam responseMessageBelowArrow true skinparam maxMessageSize 150 skinparam sequence { ParticipantBackgroundColor LightBlue ActorBackgroundColor LightGray LifeLineBorderColor gray GroupBorderColor DarkGray } title WalkGuide System — Complete Sequence Diagram ' ========================================== ' PARTICIPANTS ' ========================================== actor "Tunanetra\n(ROLE_USER)" as User actor "Pendamping\n(ROLE_GUARDIAN)" as Guardian participant "Flutter App\n(User Side)" as AppUser participant "Flutter App\n(Guardian Side)" as AppGuardian participant "Spring Boot\nBackend" as Backend participant "JWT Auth\nFilter" as JWT participant "WebSocket\n(STOMP)" as WS participant "Firebase FCM" as FCM participant "Agora RTC" as Agora participant "YOLOv8n\n(On-Device AI)" as YOLO ' ============================================================ ' 1. SERVER CONNECT ' ============================================================ == 1. Server Connect (Pertama Kali Install) == User -> AppUser : Buka aplikasi AppUser -> AppUser : Cek SharedPreferences\n(serverUrl ada?) alt serverUrl TIDAK ada AppUser -> User : Tampilkan ServerConnectScreen User -> AppUser : Input URL server\n(http://202.46.28.160:8080) AppUser -> Backend : GET /api/v1/health (ping) alt Server OK Backend --> AppUser : 200 OK AppUser -> AppUser : Simpan serverUrl\nke SharedPreferences AppUser -> User : Redirect ke LoginScreen else Server Gagal Backend --> AppUser : Timeout / Error AppUser -> User : Tampilkan pesan error\n"Server tidak dapat dijangkau" end else serverUrl sudah ada AppUser -> User : Langsung ke SplashScreen end ' ============================================================ ' 2. AUTH — REGISTER ' ============================================================ == 2. Register Akun == User -> AppUser : Isi form Register\n(email, password, role) AppUser -> Backend : POST /api/v1/auth/register\n{email, password, displayName, role} Backend -> Backend : Validasi @NotBlank, @Email Backend -> Backend : Cek email unik Backend -> Backend : BCrypt encode password alt role = ROLE_USER Backend -> Backend : Generate uniqueUserId\n(12 char alphanumeric) end Backend -> Backend : Simpan user ke DB Backend -> Backend : Buat default UserSettings Backend -> Backend : Generate JWT access token (1 jam)\n+ refresh token (30 hari) Backend -> Backend : Simpan refresh token ke DB Backend --> AppUser : 201 Created\n{accessToken, refreshToken, role, userId, uniqueUserId} AppUser -> AppUser : Simpan token ke\nFlutterSecureStorage AppUser -> Backend : PUT /api/v1/shared/fcm-token\n{fcmToken} Backend --> AppUser : 200 OK AppUser -> User : Redirect ke Home Screen\n(berdasarkan role) ' ============================================================ ' 3. AUTH — LOGIN ' ============================================================ == 3. Login == User -> AppUser : Input email & password AppUser -> Backend : POST /api/v1/auth/login\n{email, password} Backend -> Backend : findByEmail → validasi BCrypt Backend -> Backend : Generate JWT access token\n+ refresh token Backend -> Backend : Simpan refresh token ke DB Backend -> Backend : Log ActivityLog: LOGIN Backend --> AppUser : 200 OK\n{accessToken, refreshToken, role, userId} AppUser -> AppUser : Simpan token ke\nFlutterSecureStorage AppUser -> Backend : PUT /api/v1/shared/fcm-token\n{fcmToken} Backend --> AppUser : 200 OK AppUser -> WS : STOMP connect\n(Authorization: Bearer token) WS --> AppUser : Connected AppUser -> User : Redirect ke Home\n(ROLE_USER → /user/home) Guardian -> AppGuardian : Login AppGuardian -> Backend : POST /api/v1/auth/login Backend --> AppGuardian : 200 OK {accessToken, role=ROLE_GUARDIAN} AppGuardian -> WS : STOMP connect WS --> AppGuardian : Connected AppGuardian -> Guardian : Redirect ke /guardian/home ' ============================================================ ' 4. AUTH — JWT VALIDATE & REFRESH ' ============================================================ == 4. JWT Validate & Auto-Refresh == AppUser -> Backend : Request API (any endpoint)\n[Authorization: Bearer accessToken] Backend -> JWT : Validasi token alt Token valid JWT --> Backend : OK, userId + role Backend -> Backend : RBAC check role Backend --> AppUser : 200 Response else Token expired (401) JWT --> Backend : 401 Unauthorized Backend --> AppUser : 401 Unauthorized AppUser -> Backend : POST /api/v1/auth/refresh\n{refreshToken} Backend -> Backend : Cek refresh token di DB\n+ cek expired alt Refresh token valid Backend -> Backend : Generate access token baru Backend --> AppUser : 200 OK {accessToken baru} AppUser -> AppUser : Update token di SecureStorage AppUser -> Backend : Retry request original Backend --> AppUser : 200 Response else Refresh token expired Backend --> AppUser : 401 Unauthorized AppUser -> AppUser : Clear semua token AppUser -> User : Redirect ke /login end end ' ============================================================ ' 5. PAIRING ' ============================================================ == 5. Pairing — Guardian Undang User == Guardian -> AppGuardian : Input uniqueUserId\n(12 char) milik User AppGuardian -> Backend : POST /api/v1/guardian/pairing/invite\n{uniqueUserId} Backend -> Backend : Cek User exist by uniqueUserId Backend -> Backend : Cek belum ada pairing aktif Backend -> Backend : Buat PairingRelation\n(status=PENDING) Backend -> FCM : Kirim push notif ke User\n(type=PAIRING_INVITE) FCM --> AppUser : Push notification diterima AppUser -> AppUser : FCM Handler:\nrefresh PairingBloc AppUser -> AppUser : TTS: "Anda mendapat\nundangan pairing" Backend --> AppGuardian : 200 OK {pairingId, status=PENDING} User -> AppUser : Buka notif / PairingScreen AppUser -> Backend : GET /api/v1/user/pairing/pending Backend --> AppUser : 200 OK {pairingId, guardianName} User -> AppUser : Tap "Terima" atau "Tolak" AppUser -> Backend : POST /api/v1/user/pairing/respond\n{pairingId, accept: true/false} Backend -> Backend : Update PairingRelation\n(status=ACTIVE / REJECTED) alt Accept Backend -> Backend : Seed default VoiceCommandConfigs\n& HardwareShortcuts untuk pasangan ini Backend -> FCM : Kirim notif ke Guardian\n(type=PAIRING_RESPONSE, accepted=true) FCM --> AppGuardian : Push notification AppGuardian -> AppGuardian : TTS Guardian:\n"User menerima pairing Anda" Backend --> AppUser : 200 OK {status=ACTIVE} AppUser -> User : TTS: "Pairing berhasil" else Reject Backend -> FCM : Kirim notif ke Guardian\n(type=PAIRING_RESPONSE, accepted=false) Backend --> AppUser : 200 OK {status=REJECTED} end ' ============================================================ ' 6. WALK GUIDE MODE ' ============================================================ == 6. Aktifkan Walk Guide Mode == User -> AppUser : Tap tombol / ucap\n"Start Walkguide" AppUser -> AppUser : VoiceCommandHandler:\nmatch trigger phrase AppUser -> AppUser : WalkGuideBloc:\nadd(StartWalkGuide()) AppUser -> AppUser : Request permission:\nkamera, lokasi, mikrofon AppUser -> YOLO : loadModel()\n(yolov8n.tflite dari assets) YOLO --> AppUser : Model loaded AppUser -> AppUser : Aktifkan kamera stream AppUser -> AppUser : Aktifkan GPS stream\n(interval 5 detik) AppUser -> User : TTS: "Walk Guide aktif.\nDeteksi obstacle dimulai." loop Setiap frame kamera (max 5 FPS) AppUser -> YOLO : detect(CameraImage frame) YOLO -> YOLO : YUV420 → RGB → resize 640×640 YOLO -> YOLO : Normalize → run inference YOLO -> YOLO : Post-process: NMS,\nfilter confidence threshold YOLO --> AppUser : List\n{label, confidence, boundingBox} alt Ada obstacle terdeteksi AppUser -> AppUser : ObstacleAnalyzer:\nanalyzeDirection() + estimateDistance() AppUser -> AppUser : buildTtsMessage():\n"Caution! {label} {direction}. {distance}." AppUser -> AppUser : ttsService.speakImmediate(message) AppUser -> User : 🔊 Audio TTS obstacle alert AppUser -> AppUser : hapticService.obstacle\n{VeryClose/Close/Medium}() AppUser -> User : 📳 Vibrasi sesuai jarak AppUser -> Backend : POST /api/v1/user/obstacle-log\n{label, confidence, direction, dist, lat, lng} Backend --> AppUser : 200 OK end end ' ============================================================ ' 7. KIRIM LOKASI GPS REAL-TIME ' ============================================================ == 7. Kirim Lokasi GPS Real-time (WebSocket) == loop Setiap 5 detik (WalkGuide aktif) / 30 detik (idle) AppUser -> AppUser : Geolocator:\ngetCurrentPosition() AppUser -> WS : STOMP send\n/app/location/{userId}\n{lat, lng, accuracy, speed, heading} WS -> Backend : Proses LocationUpdate Backend -> Backend : Simpan ke location_history WS -> AppGuardian : Broadcast ke\n/topic/location/{userId} AppGuardian -> AppGuardian : Update marker\ndi flutter_map (real-time) AppGuardian -> Guardian : 🗺️ Peta terupdate alt User keluar radius Geofence Backend -> Backend : Haversine: cek jarak\nke center geofence Backend -> FCM : Push notif ke Guardian\n(type=GEOFENCE_EXIT) FCM --> AppGuardian : Notifikasi masuk AppGuardian -> Guardian : "User telah keluar\ndari area aman!" end end ' ============================================================ ' 8. SOS DARURAT ' ============================================================ == 8. Kirim SOS Darurat == User -> AppUser : Ucap "Send SOS" /\ntap tombol SOS AppUser -> AppUser : SosBloc: add(TriggerSos()) AppUser -> AppUser : hapticService.sosTriggered()\n[0,1000,200,1000,200,1000] AppUser -> AppUser : ttsService.speak:\n"SOS dikirim ke Guardian" AppUser -> Backend : POST /api/v1/user/sos\n{triggerType, lat, lng} Backend -> Backend : Simpan SosEvent\n(status=TRIGGERED) Backend -> Backend : Log ActivityLog: SOS_TRIGGERED Backend -> FCM : Push notif ke Guardian\n(type=SOS_ALERT, lat, lng) FCM --> AppGuardian : Push notification SOS AppGuardian -> AppGuardian : FCM Handler:\nnavigate ke SOS/IncomingCallScreen AppGuardian -> Guardian : 🚨 Alert SOS dengan lokasi User Backend --> AppUser : 200 OK {sosEventId, status=TRIGGERED} Guardian -> AppGuardian : Tap "Acknowledge" AppGuardian -> Backend : PUT /api/v1/guardian/sos/{id}/acknowledge Backend -> Backend : Update status=ACKNOWLEDGED\nsimpan acknowledged_at Backend --> AppGuardian : 200 OK AppGuardian -> Guardian : Tampilkan status\n"SOS Diakui" Guardian -> AppGuardian : Tap "Resolved" AppGuardian -> Backend : PUT /api/v1/guardian/sos/{id}/resolve Backend -> Backend : Update status=RESOLVED Backend --> AppGuardian : 200 OK ' ============================================================ ' 9. VoIP CALL (AGORA) ' ============================================================ == 9. Panggil Guardian (VoIP Agora) == User -> AppUser : Ucap "Call Guardian" /\ntap tombol Call AppUser -> Backend : POST /api/v1/shared/agora/token\n{channelName, uid} Backend -> Agora : Generate Agora RTC token\n(server-side REST API) Agora --> Backend : {token, channelName, uid, appId} Backend --> AppUser : 200 OK {AgoraTokenResponse} AppUser -> FCM : (via Backend) Push notif ke Guardian\n(type=INCOMING_CALL, channelName) FCM --> AppGuardian : Push notification panggilan masuk AppGuardian -> AppGuardian : Navigate ke IncomingCallScreen AppGuardian -> Guardian : 📞 UI Panggilan Masuk\n(haptic + ringtone) AppUser -> Agora : joinChannel(token, channelName, uid)\n(audio-only, speakerphone ON) Guardian -> AppGuardian : Tap "Angkat" AppGuardian -> Backend : POST /api/v1/shared/agora/token\n{channelName} Backend --> AppGuardian : {AgoraTokenResponse} AppGuardian -> Agora : joinChannel(token, channelName) Agora --> AppUser : onUserJoined callback Agora --> AppGuardian : onUserJoined callback AppUser <-> AppGuardian : 🎙️ Audio call berlangsung\n(via Agora RTC) alt Guardian tutup telepon Guardian -> AppGuardian : Tap "Tutup" AppGuardian -> Agora : leaveChannel() Agora --> AppUser : onUserOffline callback AppUser -> Agora : leaveChannel() AppUser -> User : TTS: "Panggilan berakhir" end ' ============================================================ ' 10. NOTIFIKASI GUARDIAN → USER ' ============================================================ == 10. Kirim Notifikasi (Teks / Voice Note) == Guardian -> AppGuardian : Buka SendNotifScreen\nTulis pesan / rekam voice note alt Notifikasi Teks AppGuardian -> Backend : POST /api/v1/guardian/notification\n{notifType=TEXT, content} else Voice Note AppGuardian -> AppGuardian : record package:\nrekam audio AppGuardian -> Backend : POST /api/v1/guardian/notification\n{notifType=VOICE_NOTE,\nvoiceNoteUrl, voiceNoteDuration} end Backend -> Backend : Simpan ke guardian_notifications Backend -> FCM : Kirim push notif ke User\n(type=NOTIFICATION) FCM --> AppUser : Push notification diterima AppUser -> AppUser : FCM Handler: type=NOTIFICATION\n→ refresh NotificationBloc AppUser -> AppUser : flutter_local_notifications:\ntampilkan notif lokal AppUser -> AppUser : TTS: "Ada pesan baru\ndari Guardian" AppUser -> User : 🔊 TTS pemberitahuan Backend --> AppGuardian : 200 OK User -> AppUser : Buka NotificationScreen\natau ucap "Read All My Notifications" AppUser -> Backend : GET /api/v1/user/notifications Backend --> AppUser : 200 OK List AppUser -> User : Tampilkan daftar notifikasi alt Voice Note User -> AppUser : Tap notifikasi voice note AppUser -> AppUser : just_audio: play voiceNoteUrl AppUser -> User : 🔊 Putar voice note Guardian end AppUser -> Backend : PUT /api/v1/user/notifications/read-all Backend -> Backend : Update is_read=true semua notif Backend --> AppUser : 200 OK ' ============================================================ ' 11. GUARDIAN DASHBOARD ' ============================================================ == 11. Guardian Dashboard — Monitor & Konfigurasi == Guardian -> AppGuardian : Buka GuardianDashboardScreen AppGuardian -> Backend : GET /api/v1/guardian/paired-user Backend --> AppGuardian : 200 OK {UserProfileResponse} AppGuardian -> Backend : GET /api/v1/guardian/location/latest Backend --> AppGuardian : 200 OK {lat, lng, speed, heading} AppGuardian -> AppGuardian : flutter_map: tampilkan\nmarker lokasi User AppGuardian -> WS : Subscribe /topic/location/{userId} AppGuardian -> Guardian : 🗺️ Dashboard dengan peta real-time Guardian -> AppGuardian : Buka GuardianAiConfigScreen AppGuardian -> Backend : GET /api/v1/guardian/ai-config Backend --> AppGuardian : 200 OK {AiConfigResponse} Guardian -> AppGuardian : Ubah threshold confidence,\ndistance, FPS, enabled labels AppGuardian -> Backend : PUT /api/v1/guardian/ai-config\n{AiConfigUpdateRequest} Backend -> Backend : Update ai_configs di DB Backend -> FCM : Push notif ke User\n(type=SETTINGS_UPDATED) FCM --> AppUser : Settings update diterima AppUser -> Backend : GET /api/v1/user/ai-config Backend --> AppUser : 200 OK {AiConfigResponse terbaru} AppUser -> YOLO : Update confidenceThreshold\n& maxInferenceFps Backend --> AppGuardian : 200 OK Guardian -> AppGuardian : Buka GuardianGeofenceScreen AppGuardian -> Backend : GET /api/v1/guardian/geofence Backend --> AppGuardian : 200 OK {GeofenceResponse} Guardian -> AppGuardian : Set center (tap peta),\nradius, enable AppGuardian -> Backend : PUT /api/v1/guardian/geofence\n{centerLat, centerLng, radiusMeters, enabled} Backend -> Backend : Update geofence_configs Backend --> AppGuardian : 200 OK AppGuardian -> Guardian : Tampilkan lingkaran\ngeofence di peta Guardian -> AppGuardian : Buka GuardianVoiceCmdScreen AppGuardian -> Backend : GET /api/v1/guardian/voice-commands Backend --> AppGuardian : 200 OK List Guardian -> AppGuardian : Ubah trigger phrase\n(misal: "mulai jalan") AppGuardian -> Backend : PUT /api/v1/guardian/voice-commands\n{commandKey, triggerPhrase, enabled} Backend -> Backend : Update voice_command_configs Backend -> FCM : Push notif ke User\n(type=SETTINGS_UPDATED) FCM --> AppUser : AppUser reload CachedVoiceCommands\nke SQLite lokal Backend --> AppGuardian : 200 OK Guardian -> AppGuardian : Buka GuardianActivityLogScreen AppGuardian -> Backend : GET /api/v1/guardian/activity-logs?page=0&size=20 Backend --> AppGuardian : 200 OK Page AppGuardian -> Backend : GET /api/v1/guardian/obstacle-logs?page=0&size=20 Backend --> AppGuardian : 200 OK Page AppGuardian -> Guardian : Tampilkan riwayat\naktivitas & deteksi obstacle ' ============================================================ ' 12. LOGOUT ' ============================================================ == 12. Logout == User -> AppUser : Tap Logout AppUser -> Backend : POST /api/v1/auth/logout Backend -> Backend : Delete refresh token dari DB Backend -> Backend : Log ActivityLog: LOGOUT Backend --> AppUser : 200 OK AppUser -> WS : disconnect() AppUser -> AppUser : SecureStorage.clearAll() AppUser -> AppUser : SharedPreferences: hapus token AppUser -> User : Redirect ke /login @enduml