From c5d716248ea798a1c80aba68bf78de8b8aeb3de1 Mon Sep 17 00:00:00 2001 From: Wowieee4 Date: Mon, 4 May 2026 11:49:18 +0700 Subject: [PATCH] update readme --- README.md | 426 ++++++++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 362 insertions(+), 64 deletions(-) diff --git a/README.md b/README.md index 0b6b24e..2d2b8a0 100644 --- a/README.md +++ b/README.md @@ -23,12 +23,14 @@ [![Flutter](https://img.shields.io/badge/Flutter-Clean_Architecture-02569B?style=flat-square&logo=flutter&logoColor=white)](https://flutter.dev/) [![Spring Boot](https://img.shields.io/badge/Spring_Boot-REST_API-6DB33F?style=flat-square&logo=spring&logoColor=white)](https://spring.io/projects/spring-boot) [![PostgreSQL](https://img.shields.io/badge/PostgreSQL-Database-4169E1?style=flat-square&logo=postgresql&logoColor=white)](https://www.postgresql.org/) +[![Firebase](https://img.shields.io/badge/Firebase-FCM-FFCA28?style=flat-square&logo=firebase&logoColor=black)](https://firebase.google.com/) +[![Agora](https://img.shields.io/badge/Agora-VoIP-099DFD?style=flat-square)](https://www.agora.io/) [![License](https://img.shields.io/badge/License-MIT-22c55e?style=flat-square)](LICENSE) ![Last Updated](https://img.shields.io/badge/Last_Updated-WIP-blue?style=for-the-badge) ![Author](https://img.shields.io/badge/By-evan--william-brightgreen?style=for-the-badge) -[**System Architecture**](#system-architecture) · [**Implementations**](#implementations) · [**Results**](#results) · [**Weekly Progress**](#weekly-progress) +[**System Architecture**](#system-architecture) · [**Tech Stack**](#tech-stack) · [**Implementations**](#implementations) · [**API Endpoints**](#api-endpoints) · [**Design Patterns**](#design-patterns) · [**Results**](#results) · [**Weekly Progress**](#weekly-progress) @@ -38,7 +40,9 @@ **Core Objective:** How can we build an ultra-low latency, accessible navigation system for visually impaired users while providing real-time oversight for their guardians? -This project implements a dual-interface mobile application. The system relies on **On-Device AI (TFLite)** to eliminate network latency during obstacle detection, paired with a robust **Spring Boot backend** for secure authentication and guardian-user pairing. +This project implements a dual-interface mobile application. The system relies on **On-Device AI (TFLite / YOLOv8n)** to eliminate network latency during obstacle detection, paired with a robust **Spring Boot backend** for secure authentication, guardian-user pairing, real-time location tracking via WebSocket, and push notifications via Firebase FCM. + +**Deployment:** Backend is deployed on a university server at `202.46.28.160`. The Flutter APK uses a dynamic server URL — no hardcoded addresses — allowing multi-device testing without rebuilding. --- @@ -46,11 +50,68 @@ This project implements a dual-interface mobile application. The system relies o The study follows a strict three-pillar enterprise structure: -**Pillar 1 — OOAD (Object-Oriented Analysis & Design):** Comprehensive modeling using Use Case, Class, Sequence, and ERD diagrams. The codebase strictly implements ≥ 4 GoF Design Patterns (e.g., Builder, Singleton, Repository, Strategy). +**Pillar 1 — OOAD (Object-Oriented Analysis & Design):** Comprehensive modeling using Use Case, Class, Sequence, and ERD diagrams. The codebase strictly implements ≥ 7 GoF Design Patterns (Builder, Singleton, Facade, Repository/Proxy, Observer, Strategy, Chain of Responsibility). -**Pillar 2 — Flutter (Mobile Frontend):** Implements Clean Architecture (Domain, Data, Presentation layers) with BLoC/Riverpod for state management. Uses `Dio` for secure HTTP communication. +**Pillar 2 — Flutter (Mobile Frontend):** Implements Clean Architecture (Domain, Data, Presentation layers) with BLoC for state management. Uses `Dio` with interceptors for secure HTTP communication. Server URL is dynamically configured via `SharedPreferences` on first launch. -**Pillar 3 — Spring Boot (Backend API):** A layered architecture (Controller → Service → Repository) powered by Java 17+. Features JWT-based Role-Based Access Control (RBAC) and standardized `ApiResponse` envelopes. +**Pillar 3 — Spring Boot (Backend API):** A layered architecture (Controller → Service → Repository) powered by Java 21. Features JWT-based Role-Based Access Control (RBAC), standardized `ApiResponse` envelopes, WebSocket (STOMP) for real-time data, and Firebase Admin SDK for push notifications. + +--- + +## Tech Stack + +### Backend (Spring Boot) + +| Library / Tool | Version | Purpose | +|---|---|---| +| Java | 21 | Primary language | +| Spring Boot | 3.3.x | Main framework | +| Spring Security | (bundled) | Auth + RBAC | +| Spring Data JPA | (bundled) | ORM | +| Spring WebSocket (STOMP) | (bundled) | Real-time location & notification push | +| PostgreSQL Driver | (bundled) | DB connection — university server `202.46.28.160` | +| Flyway | 10.x | Database schema migration | +| JJWT | 0.11.5 | JWT access + refresh token | +| Firebase Admin SDK | 9.x | FCM push notifications | +| Agora RESTful API | - | Generate Agora RTC token for VoIP | +| Springdoc OpenAPI | 2.3.0 | Swagger UI documentation | +| Lombok | latest | Boilerplate reduction | +| JUnit 5 + Mockito | (bundled) | Unit testing | +| MockMvc + Testcontainers | 1.19.x | Integration testing with real PostgreSQL | +| JaCoCo | 0.8.x | Code coverage (target ≥ 70%) | + +### Flutter (Mobile) + +| Package | Version | Purpose | +|---|---|---| +| flutter_bloc | 8.x | State management (sole pattern) | +| go_router | 14.x | Navigation + role-based route guards | +| dio | 5.x | HTTP client with interceptors | +| shared_preferences | 2.x | Persist dynamic server URL | +| flutter_secure_storage | 10.x | Secure JWT token storage | +| drift | 2.x | SQLite ORM for offline cache | +| tflite_flutter | 0.10.x | Run YOLOv8n on-device | +| camera | 0.10.x | Camera feed for YOLO inference | +| flutter_tts | 4.x | Text-to-Speech (ID + EN) | +| speech_to_text | 6.x | Always-listening voice commands | +| agora_rtc_engine | 6.x | VoIP call Guardian ↔ User | +| firebase_messaging | 14.x | FCM push notification receiver | +| flutter_map | 6.x | OpenStreetMap (free, no API key) | +| geolocator | 11.x | Real-time GPS | +| stomp_dart_client | 2.x | WebSocket STOMP for live tracking | +| get_it | 7.x | Service locator / dependency injection | +| dartz | 0.10.x | `Either` typed error handling | +| vibration | 1.x | Haptic feedback on obstacle detection | + +### External Services + +| Service | Purpose | Cost | +|---|---|---| +| Firebase (FCM) | Push notifications | Free | +| Agora RTC | In-app VoIP audio call | 10,000 min/month free | +| University Server PostgreSQL | DB at `202.46.28.160:2002` | Free (managed by lecturer) | +| OpenStreetMap + OSRM | Map tiles + turn-by-turn routing | Free | +| YOLOv8n (Ultralytics) | Obstacle detection model (.tflite) | Free (open source) | --- @@ -58,134 +119,371 @@ The study follows a strict three-pillar enterprise structure: ### Design A — User Mode (Visually Impaired) An accessibility-first interface that opens directly to an active camera view. -- **On-Device AI:** Obstacle detection using TFLite runs locally to ensure zero network latency. -- **Hardware Mapping:** Physical volume buttons mapped to critical actions (e.g., auto-accept guardian calls). -- **Voice Commands:** Integrated TTS and speech recognition for hands-free navigation. +- **On-Device AI:** YOLOv8n obstacle detection via TFLite runs locally — zero network latency. Detects 80 COCO object classes, reports direction (LEFT/CENTER/RIGHT) and estimated distance. +- **Voice Commands:** 14 configurable voice commands (e.g., "Start Walkguide", "Call Guardian", "Send SOS", "Where Am I"). Always-listening `SpeechToText` with auto-restart. +- **Hardware Mapping:** Physical volume buttons mapped to critical actions (Vol Up = Call Guardian, Vol Down = Start WalkGuide by default). +- **TTS Feedback:** Queued and immediate Text-to-Speech for obstacle alerts, navigation, and screen announcements. Supports Bahasa Indonesia and English. +- **SOS System:** One-command emergency alert triggers high-priority FCM to Guardian with GPS coordinates. -### Design B — Guardian Mode (Admin) -A command center dashboard for oversight and configuration. -- **Real-time Monitoring:** Tracks user status and device battery. -- **Remote Configuration:** Adjusts AI sensitivity and sets hardware shortcuts for the paired User device. -- **Quick Actions:** One-tap voice conferencing and emergency overrides. +### Design B — Guardian Mode (Admin/Caregiver) +A command center dashboard for oversight and remote configuration. +- **Real-time Live Map:** Tracks User's GPS location via WebSocket STOMP subscription, rendered on OpenStreetMap with `flutter_map`. Includes geofence circle overlay. +- **Remote AI Configuration:** Adjusts YOLO confidence threshold, alert distance thresholds, max inference FPS, and enabled object labels. +- **Voice Command Management:** Edits trigger phrases and enables/disables any of the 14 voice commands remotely. +- **Notifications:** Sends text messages or voice notes (recorded audio) directly to the User's device. +- **Geofence:** Sets a geographic boundary — backend notifies Guardian via FCM when User exits. + +--- + +## Database Schema + +Database runs on PostgreSQL at `202.46.28.160:2002`. Schema is managed exclusively by Flyway migrations. `spring.jpa.hibernate.ddl-auto=validate`. + +| Migration | Table | Status | +|---|---|---| +| V1 | `users` | ✅ Exists | +| V2 | Seed data | ✅ Exists | +| V3 | Guardian-User link | ✅ Exists | +| V4 | Add `unique_user_id` column | 🔄 Needs creation | +| V5 | `pairing_relations` | 🔄 Needs creation | +| V6 | `activity_logs` | 🔄 Needs creation | +| V7 | `obstacle_logs` | 🔄 Needs creation | +| V8 | `location_history` | 🔄 Needs creation | +| V9 | `guardian_notifications` | 🔄 Needs creation | +| V10 | `sos_events` | 🔄 Needs creation | +| V11 | `user_settings` | 🔄 Needs creation | +| V12 | `ai_configs` | 🔄 Needs creation | +| V13 | `voice_command_configs` | 🔄 Needs creation | +| V14 | `hardware_shortcuts` | 🔄 Needs creation | +| V15 | `geofence_configs` | 🔄 Needs creation | +| V16 | `refresh_tokens` | 🔄 Needs creation | + +--- + +## API Endpoints + +26 REST endpoints across 5 groups (exam minimum: 10). + +**Auth** — `/api/v1/auth` + +| Method | Path | Auth | Description | +|---|---|---|---| +| GET | `/ping` | None | Health check / server connection test | +| POST | `/register` | None | Register Guardian or User | +| POST | `/login` | None | Login, returns access + refresh token | +| POST | `/refresh` | None | Refresh access token | +| POST | `/logout` | ✅ | Logout, invalidate refresh token | +| PUT | `/fcm-token` | ✅ | Update FCM device token | + +**Pairing** — `/api/v1/shared/pairing` + +| Method | Path | Auth | Description | +|---|---|---|---| +| POST | `/invite` | GUARDIAN | Invite User by 12-char unique ID | +| POST | `/respond` | USER | Accept or reject pairing invite | +| DELETE | `/unpair` | ✅ | Dissolve active pairing | +| GET | `/status` | ✅ | Get current pairing status | + +**Guardian** — `/api/v1/guardian` + +| Method | Path | Description | +|---|---|---| +| GET | `/dashboard` | Combined home data (user status, recent logs, SOS count) | +| GET | `/user-status` | Full status of paired User | +| GET | `/user-location` | Last known GPS location | +| GET | `/location-history` | Paginated location history | +| GET | `/activity-logs` | Paginated activity logs of paired User | +| GET | `/obstacle-logs` | Paginated obstacle detection logs | +| POST | `/notifications/send` | Send text or voice note to User | +| GET | `/sos-events` | Paginated SOS events | +| PUT | `/sos/{id}/acknowledge` | Acknowledge SOS alert | +| GET/PUT | `/ai-config` | Get or update AI configuration | +| GET/PUT | `/voice-commands` | Get or update voice command configs | +| GET/PUT | `/shortcuts` | Get or update hardware shortcut configs | +| GET/PUT | `/geofence` | Get or update geofence config | + +**User** — `/api/v1/user` + +| Method | Path | Description | +|---|---|---| +| GET | `/profile` | User profile (id, email, displayName, uniqueUserId) | +| GET/PUT | `/settings` | Get or update TTS/haptic settings | +| GET | `/voice-commands` | Get all voice commands (read-only) | +| GET/PUT | `/shortcuts` | Get or capture hardware shortcut assignments | +| GET | `/ai-config` | Get AI config (read-only) | +| POST | `/location` | Send GPS update | +| POST | `/obstacle` | Log detected obstacle | +| POST | `/sos` | Trigger SOS alert | +| GET | `/activity-logs` | Paginated own activity log | +| GET | `/notifications` | Paginated notifications from Guardian | +| GET | `/notifications/unread-count` | Unread notification count | +| PUT | `/notifications/mark-all-read` | Mark all as read | +| PUT | `/notifications/{id}/read` | Mark one as read | +| POST | `/walkguide/start` | Log WalkGuide session start | +| POST | `/walkguide/stop` | Log WalkGuide session stop | + +**Shared Call** — `/api/v1/shared/call` + +| Method | Path | Description | +|---|---|---| +| POST | `/token` | Generate Agora RTC token | +| POST | `/notify` | Send "Incoming Call" FCM to other party | + +--- + +## Design Patterns + +7 GoF Design Patterns implemented (exam minimum: 4, minimum 1 per category). + +| # | Category | Pattern | Location | +|---|---|---|---| +| 1 | Creational | **Builder** | `User.java` (`@Builder`), `FcmService` message construction | +| 2 | Creational | **Singleton** | Flutter: `TtsService`, `YoloDetector`, `WebSocketService`, `AgoraService` via GetIt | +| 3 | Structural | **Facade** | Flutter: `VoiceCommandHandler`; Backend: `GuardianDashboardService` | +| 4 | Structural | **Repository (Proxy)** | All `*_repository_impl.dart` — proxy between domain and data sources | +| 5 | Behavioral | **Observer** | BLoC pattern (BLoC = Subject, Widgets = Observers); WebSocket callbacks | +| 6 | Behavioral | **Strategy** | `ObstacleAnalyzer` direction strategy; `ObstacleAlertStrategyService` (backend) | +| 7 | Behavioral | **Chain of Responsibility** | Spring Security filter chain; Dio interceptor chain | --- ## Metrics -| Metric | Instrument | Target Threshold | -|--------|-----------|------| +| Metric | Instrument | Target | +|---|---|---| | API Throughput | Apache JMeter / k6 | ≥ 100 req/s under load | | API p95 Latency | Apache JMeter / k6 | < 500ms | | UI Frame Rate | Flutter DevTools | ≥ 90% frames < 16ms | | Code Coverage | JaCoCo | ≥ 70% (Service & Controller) | - ---- - -## Technical Parameters - -**Backend Phase (Spring Boot):** -- Environment: Java 17+, PostgreSQL / H2 Database, Flyway Migrations. -- Security: Spring Security, JWT (Access + Refresh Tokens). -- Documentation: OpenAPI 3.0 (Swagger UI). - -**Mobile Phase (Flutter):** -- Environment: Flutter 3.x, physical Android device testing (Profile Mode). -- Storage: `flutter_secure_storage` for JWT. -- Routing: GoRouter with role-based authenticated routes. +| APK Size | `flutter build apk --analyze-size` | < 50MB | --- ## Repository Structure -> 🚨 **DISCLAIMER:** This repository is in active development. The directory tree below is a **conceptual reference snapshot** of the intended architecture. It **does not** represent the exact, current state of the files. +> 🚨 **DISCLAIMER:** This repository is in active development. The directory tree below reflects the intended final architecture. Flutter implementation is currently pending — backend (Spring Boot) is the active development phase. ```text / -├── walkguide-backend/ # [Spring Boot Engine] +├── walkguide-backend/ # [Spring Boot Engine — ACTIVE] │ ├── src/main/java/com/walkguide/ -│ │ ├── config/ # Security & Swagger configs -│ │ ├── controller/ # REST Endpoints -│ │ ├── dto/ # AuthRequest & ApiResponse -│ │ ├── entity/ # JPA Entities -│ │ └── exception/ # GlobalExceptionHandler -│ └── pom.xml +│ │ ├── config/ # SecurityConfig, WebSocketConfig, OpenApiConfig, FcmConfig, DataSeeder +│ │ ├── enums/ # UserRole, PairingStatus, ActivityLogType, VoiceCommandKey, +│ │ │ # HardwareShortcutKey, NotificationType, SosStatus +│ │ ├── entity/ # User, PairingRelation, ActivityLog, ObstacleLog, +│ │ │ # LocationHistory, GuardianNotification, SosEvent, +│ │ │ # UserSettings, AiConfig, VoiceCommandConfig, +│ │ │ # HardwareShortcut, GeofenceConfig, RefreshToken +│ │ ├── repository/ # JPA repositories for all entities +│ │ ├── dto/ +│ │ │ ├── request/ # RegisterRequest, LoginRequest, InviteUserRequest, +│ │ │ │ # LocationUpdateRequest, ObstacleLogRequest, +│ │ │ │ # SendNotificationRequest, SosRequest, +│ │ │ │ # AiConfigUpdateRequest, VoiceCommandUpdateRequest, +│ │ │ │ # HardwareShortcutUpdateRequest, GeofenceConfigRequest, +│ │ │ │ # UserSettingsUpdateRequest, and more +│ │ │ └── response/ # ApiResponse (standard wrapper), AuthDataResponse, +│ │ │ # UserProfileResponse, PairingStatusResponse, +│ │ │ # ActivityLogResponse, ObstacleLogResponse, +│ │ │ # LocationResponse, NotificationResponse, +│ │ │ # SosEventResponse, AgoraTokenResponse, +│ │ │ # AiConfigResponse, VoiceCommandResponse, +│ │ │ # HardwareShortcutResponse, GeofenceResponse +│ │ ├── service/ # AuthService, PairingService, ActivityLogService, +│ │ │ # LocationService, ObstacleLogService, +│ │ │ # NotificationService, SosService, +│ │ │ # AiConfigService, VoiceCommandService, +│ │ │ # HardwareShortcutService, GeofenceService, +│ │ │ # UserSettingsService, FcmService, +│ │ │ # AgoraTokenService, GuardianDashboardService +│ │ ├── controller/ # AuthController, PairingController, +│ │ │ # GuardianController, UserController, CallController +│ │ ├── security/ # JwtUtil, JwtAuthFilter, CustomUserDetailsService +│ │ ├── websocket/ # LocationBroadcaster (STOMP broadcaster) +│ │ └── exception/ # GlobalExceptionHandler, ResourceNotFoundException, +│ │ # UnauthorizedException, PairingException +│ ├── src/main/resources/ +│ │ ├── application.properties # DB config (PostgreSQL 202.46.28.160:2002), Flyway, JWT +│ │ ├── firebase/ +│ │ │ └── google-services-admin.json # FCM service account key (gitignored!) +│ │ └── db/migration/ +│ │ ├── V1__create_users_table.sql ✅ Exists +│ │ ├── V2__seed_users.sql ✅ Exists +│ │ ├── V3__link_guardian_user.sql ✅ Exists +│ │ ├── V4__add_unique_user_id.sql 🔄 Needs creation +│ │ ├── V5__create_pairing_relations.sql 🔄 Needs creation +│ │ ├── V6__create_activity_logs.sql 🔄 Needs creation +│ │ ├── V7__create_obstacle_logs.sql 🔄 Needs creation +│ │ ├── V8__create_location_history.sql 🔄 Needs creation +│ │ ├── V9__create_notifications.sql 🔄 Needs creation +│ │ ├── V10__create_sos_events.sql 🔄 Needs creation +│ │ ├── V11__create_user_settings.sql 🔄 Needs creation +│ │ ├── V12__create_ai_configs.sql 🔄 Needs creation +│ │ ├── V13__create_voice_commands.sql 🔄 Needs creation +│ │ ├── V14__create_hardware_shortcuts.sql 🔄 Needs creation +│ │ ├── V15__create_geofence_configs.sql 🔄 Needs creation +│ │ └── V16__create_refresh_tokens.sql 🔄 Needs creation +│ └── src/test/java/com/walkguide/ +│ ├── service/ # AuthServiceTest, PairingServiceTest, +│ │ # NotificationServiceTest, SosServiceTest, +│ │ # LocationServiceTest, AiConfigServiceTest +│ └── controller/ # AuthControllerTest, GuardianControllerTest, +│ # UserControllerTest (MockMvc + Testcontainers) │ -├── walkguide-mobile/ # [Flutter Frontend] +├── walkguide-mobile/ # [Flutter Frontend — PENDING] │ ├── lib/ -│ │ ├── core/ # API Service, Storage, Failures +│ │ ├── main.dart # Firebase init, GetIt setup, camera init +│ │ ├── app/ +│ │ │ ├── app.dart # MultiBlocProvider + MaterialApp.router +│ │ │ ├── router.dart # GoRouter: /server-connect → /splash → role-based routes +│ │ │ └── injection_container.dart # GetIt singletons & factories +│ │ ├── core/ +│ │ │ ├── constants/ # app_constants.dart (dynamic BASE_URL via SharedPreferences), +│ │ │ │ # route_constants.dart, voice_command_keys.dart +│ │ │ ├── errors/ # failures.dart, exceptions.dart +│ │ │ ├── network/ # api_client.dart (Dio), auth_interceptor.dart, +│ │ │ │ # error_interceptor.dart, network_info.dart +│ │ │ ├── storage/ # secure_storage.dart (JWT), local_database.dart (Drift) +│ │ │ ├── services/ # tts_service.dart, stt_service.dart, +│ │ │ │ # voice_command_handler.dart, +│ │ │ │ # hardware_shortcut_listener.dart, +│ │ │ │ # fcm_service.dart, haptic_service.dart, +│ │ │ │ # websocket_service.dart, agora_service.dart +│ │ │ ├── ai/ # model_loader.dart, yolo_detector.dart, +│ │ │ │ # obstacle_analyzer.dart +│ │ │ ├── theme/ # app_theme.dart, app_colors.dart +│ │ │ └── utils/ # permission_manager.dart, location_service.dart │ │ ├── features/ -│ │ │ ├── auth/ # Clean Arch: domain, data, presentation -│ │ │ └── home/ # User & Guardian Dashboards -│ │ └── main.dart +│ │ │ ├── server_connect/ # ServerConnectScreen — FIRST SCREEN on fresh install +│ │ │ │ # BLoC: TestConnection, SaveAndContinue, ChangeServer +│ │ │ ├── auth/ # SplashScreen, LoginScreen, RegisterScreen +│ │ │ │ # Clean Arch: domain / data / presentation +│ │ │ ├── pairing/ # UserPairingScreen (show unique ID, accept invite) +│ │ │ │ # GuardianPairingScreen (invite by 12-char ID) +│ │ │ ├── walk_guide/ # WalkGuideScreen (full-screen camera + YOLO overlay) +│ │ │ │ # WalkGuideBloc: Start → camera stream → detect → TTS +│ │ │ ├── sos/ # SosScreen (large SOS button) +│ │ │ ├── activity_log/ # ActivityLogScreen (paginated, filterable) +│ │ │ ├── notifications/ # NotificationScreen + TTS read-all feature +│ │ │ ├── navigation_mode/ # NavigationModeScreen (OSM map + OSRM routing) +│ │ │ ├── settings/ # UserSettingsScreen (TTS, pairing, account) +│ │ │ ├── call/ # CallScreen + IncomingCallScreen (Agora VoIP) +│ │ │ ├── manual/ # ManualScreen (all voice commands & shortcuts) +│ │ │ └── guardian_dashboard/ # GuardianDashboardScreen, GuardianMapScreen, +│ │ │ # GuardianActivityLogScreen, GuardianSendNotifScreen, +│ │ │ # GuardianAiConfigScreen, GuardianVoiceCmdScreen, +│ │ │ # GuardianShortcutScreen, GuardianGeofenceScreen +│ │ └── shared/widgets/ # AppBottomNav, VoiceCommandListener, +│ │ # TtsAwareScaffold, LoadingOverlay, ErrorWidget +│ ├── assets/ +│ │ ├── models/ +│ │ │ ├── yolov8n.tflite # YOLOv8n converted to TFLite (~6MB) +│ │ │ └── labels.txt # 80 COCO object labels +│ │ └── images/ │ └── pubspec.yaml │ -├── ooad-docs/ # [Design Artifacts] -│ ├── diagrams/ # PlantUML / draw.io exports +├── ooad-docs/ # [Design Artifacts] +│ ├── diagrams/ # PlantUML / draw.io exports │ └── Traceability_Matrix.md │ └── README.md -```` +``` ------ +--- + +## Feature Flows (High-Level) + +### Flow 1 — Register & Pairing +Guardian registers → User registers (gets a 12-char `uniqueUserId`) → Guardian enters User's ID → backend sends FCM invite → User accepts in app → backend seeds 14 default voice commands, 5 hardware shortcuts, and AI config for the pair. + +### Flow 2 — WalkGuide Active Detection +User says "Start Walkguide" (or presses Volume Down) → camera starts → frames are throttled to max inference FPS → YOLOv8n detects obstacles → direction and distance are analyzed → TTS announces "Caution! Person ahead center. Very close. Please stop." → haptic feedback triggers → obstacle logged to backend → GPS location broadcast via WebSocket to Guardian's live map every 5 seconds. + +### Flow 3 — VoIP Call +User says "Call Guardian" → Agora token fetched from backend → FCM "Incoming Call" sent to Guardian → Guardian accepts in `IncomingCallScreen` → both join same Agora RTC channel → audio call connects. + +### Flow 4 — Guardian Sends Notification → User Hears via TTS +Guardian sends text/voice note → backend saves notification + sends FCM → User receives in foreground via WebSocket → `flutter_local_notifications` shown + TTS announced → User says "Read All My Notifications" → TTS reads each message → marked as read. + +### Flow 5 — SOS Alert +User says "Send SOS" (or hardware shortcut) → GPS captured → backend saves SOS event → high-priority FCM sent to Guardian → Guardian map opens to User's location → Guardian taps Acknowledge → FCM sent back to User → TTS "Guardian is on the way." + +### Flow 6 — Geofence +Guardian sets center + radius on map → User's location updates checked against Haversine distance → on exit: FCM to Guardian + `GEOFENCE_EXIT` activity log. + +--- ## Quick Start (WIP) ```bash # 1. Clone the repository -git clone [https://github.com/YourGroup/walkguide-final-exam.git](https://github.com/YourGroup/walkguide-final-exam.git) +git clone https://github.com/YourGroup/walkguide-final-exam.git # 2. Run Backend (Spring Boot) cd walkguide-backend ./mvnw spring-boot:run +# Flyway auto-migrates V1–V16 on first start +# Swagger UI: http://202.46.28.160:8080/swagger-ui.html +# Health check: GET http://202.46.28.160:8080/api/v1/auth/ping -# 3. Run Mobile (Flutter) - Connect to local backend +# 3. Run Mobile (Flutter) - Connect to local or university backend cd ../walkguide-mobile flutter pub get flutter run +# On first launch: enter server URL in ServerConnectScreen +# Example: http://202.46.28.160:8080 ``` ------ +--- ## Results > ⏳ **Work In Progress:** Results are populated as benchmarking phases are completed. | Metric | Baseline | Final Optimized | Status | -|--------|-----------------|----------------------|---------| +|---|---|---|---| | Cold Start Time | *Pending* | *Pending* | ⏳ | -| Memory Leak (10 Navs)| *Pending* | *Pending* | ⏳ | +| Memory Leak (10 Navs) | *Pending* | *Pending* | ⏳ | | API Error Rate | *Pending* | *Pending* | ⏳ | +| YOLO Inference Latency (ms) | *Pending* | *Pending* | ⏳ | +| API p95 Latency | *Pending* | *Pending* | ⏳ | +| JaCoCo Coverage | *Pending* | *Pending* | ⏳ | ------ +--- ## Weekly Progress | Week | Target | Status | -|------|--------|--------| +|---|---|---| | 1 | Topic proposal, Use Case definitions, Repo setup | ✅ Done | -| 2-3 | OOAD diagrams, OpenAPI YAML drafted | 🔄 In Progress | -| 4 | Clean Architecture & Spring Boot layers implemented | ⏳ Pending | -| 5 | Mid-sprint benchmarking (Flutter + Backend) | ⏳ Pending | -| 6-7 | Feature freeze, AI integration, Integration testing | ⏳ Pending | -| 8 | Final benchmarks, Report writing, Demo Video | ⏳ Pending | +| 2–3 | OOAD diagrams, OpenAPI YAML drafted | 🔄 In Progress | +| 4 | Spring Boot: Auth, Pairing, Entity, Migration V4–V16 | 🔄 In Progress | +| 5 | Spring Boot: Location, SOS, Notification, WS, FCM, Agora | ⏳ Pending | +| 6 | Spring Boot: Unit + Integration tests, JaCoCo ≥ 70% | ⏳ Pending | +| 7 | Flutter: ServerConnect, Auth, WalkGuide + YOLO pipeline | ⏳ Pending | +| 8 | Flutter: Guardian dashboard, Call, SOS, Notifications | ⏳ Pending | +| 9 | Feature freeze, integration testing, benchmark on device | ⏳ Pending | +| 10 | Final benchmarks, Report writing, Demo Video | ⏳ Pending | ------ +--- ## Core Objectives -**O₁ (Performance):** The dual-platform architecture must maintain a 60fps UI frame rate while handling concurrent backend requests without exceeding 500ms latency. +**O₁ (Performance):** The dual-platform architecture must maintain a 60fps UI frame rate while handling concurrent backend requests without exceeding 500ms latency. YOLO inference runs on a separate isolate to avoid blocking the UI thread. -**O₂ (Accessibility):** The User mode must require zero visual interaction post-login, relying entirely on haptics, voice, and physical button mapping. +**O₂ (Accessibility):** The User mode must require zero visual interaction post-login, relying entirely on haptics, voice commands, and physical button mapping. All screen transitions are announced via TTS. **O₃ (Traceability):** Every major feature implementation must be directly traceable back to the pre-development OOAD artifacts and GoF design patterns. ------ +**O₄ (Configurability):** All AI sensitivity settings, voice command phrases, and hardware shortcuts are remotely configurable by the Guardian without requiring an app update. + +--- ## License -Distributed under the [MIT License](https://www.google.com/search?q=LICENSE). +Distributed under the [MIT License](LICENSE). ------ +--- *Final Exam: Integrated Mobile Application Project*
*Flutter × Spring Boot × Object-Oriented Analysis and Design*