diff --git a/README.md b/README.md index 2d2b8a0..ae89426 100644 --- a/README.md +++ b/README.md @@ -1,489 +1,523 @@ -
- -> ⚠️ **WORK IN PROGRESS (WIP)** ⚠️ -> *This repository is currently under active development. All code, data, and structures are subject to continuous changes.* - -
- -WalkGuide Banner - # WalkGuide: AI-Powered Navigation -**Integrated Mobile Application Project** -*Flutter Mobile Frontend × Spring Boot Backend × OOAD* +Integrated Mobile Application Project +Flutter Mobile Frontend x Spring Boot Backend x OOAD -### Group Members +WalkGuide is an accessibility-focused mobile system for visually impaired users and their guardians. The User app provides camera-based obstacle awareness, voice/TTS interaction, SOS, notifications, navigation, and call flows. The Guardian app provides monitoring, live location, SOS handling, remote configuration, notification sending, voice notes, and pairing management. + +## Group Members | Name | NIM | Responsibility | -|------|-----|---------------| -| Bambang Herlambang | 5803024019 | - | -| Jap Robertus | 5803024004 | - | -| Evan William | 5803024001 | Backend Engineer (Spring Boot API & Flutter) | +|---|---:|---| +| Bambang Herlambang | 5803024019 | Mobile feature support, documentation, testing | +| Jap Robertus | 5803024004 | Mobile feature support, documentation, testing | +| Evan William | 5803024001 | Backend API, Flutter integration, architecture alignment | -[![Flutter](https://img.shields.io/badge/Flutter-Clean_Architecture-02569B?style=flat-square&logo=flutter&logoColor=white)](https://flutter.dev/) -[![Spring Boot](https://img.shields.io/badge/Spring_Boot-REST_API-6DB33F?style=flat-square&logo=spring&logoColor=white)](https://spring.io/projects/spring-boot) -[![PostgreSQL](https://img.shields.io/badge/PostgreSQL-Database-4169E1?style=flat-square&logo=postgresql&logoColor=white)](https://www.postgresql.org/) -[![Firebase](https://img.shields.io/badge/Firebase-FCM-FFCA28?style=flat-square&logo=firebase&logoColor=black)](https://firebase.google.com/) -[![Agora](https://img.shields.io/badge/Agora-VoIP-099DFD?style=flat-square)](https://www.agora.io/) -[![License](https://img.shields.io/badge/License-MIT-22c55e?style=flat-square)](LICENSE) +## Project Status -![Last Updated](https://img.shields.io/badge/Last_Updated-WIP-blue?style=for-the-badge) -![Author](https://img.shields.io/badge/By-evan--william-brightgreen?style=for-the-badge) +This repository is no longer a backend-only skeleton. The current codebase contains: -[**System Architecture**](#system-architecture) · [**Tech Stack**](#tech-stack) · [**Implementations**](#implementations) · [**API Endpoints**](#api-endpoints) · [**Design Patterns**](#design-patterns) · [**Results**](#results) · [**Weekly Progress**](#weekly-progress) +- Spring Boot backend with JWT/RBAC, Flyway migrations V1-V17, PostgreSQL, OpenAPI, WebSocket, FCM service hooks, Agora token/call notification flow, SOS acknowledge/resolve flow, pairing code flow, service/controller tests, Testcontainers setup, and k6 assets. +- Flutter app with server connection screen, auth, role-based routing, User screens, Guardian screens, feature page UI shell, friendly error handling, offline queue/cache layer, voice note UI support, SOS handling UI, WalkGuide/YOLO support files, and Android/mobile-first dependencies. +- OOAD documentation in `ooad-docs/`, including the 7 GoF design pattern PUML diagrams and traceability documentation. -
+Primary demo target: Android APK connected to the Spring Boot backend. +Chrome/web can be used for UI/debug flows, but camera, native AI, SQLite FFI, and mobile permissions are Android-first. ---- +## Overview -## Overview — WalkGuide System +Core objective: build an accessible navigation assistant that can help visually impaired users move more safely while allowing a Guardian to monitor, configure, and respond to events in real time. -**Core Objective:** How can we build an ultra-low latency, accessible navigation system for visually impaired users while providing real-time oversight for their guardians? +Important flows implemented or represented in the codebase: -This project implements a dual-interface mobile application. The system relies on **On-Device AI (TFLite / YOLOv8n)** to eliminate network latency during obstacle detection, paired with a robust **Spring Boot backend** for secure authentication, guardian-user pairing, real-time location tracking via WebSocket, and push notifications via Firebase FCM. - -**Deployment:** Backend is deployed on a university server at `202.46.28.160`. The Flutter APK uses a dynamic server URL — no hardcoded addresses — allowing multi-device testing without rebuilding. - ---- +- Register/Login with Guardian and User roles. +- Pairing by generated pairing code / user identity flow. +- Guardian dashboard and tools screens. +- User SOS trigger, Guardian acknowledge and resolve. +- Guardian text and voice-note style notification sending. +- Notification read/read-all handling. +- Location update and Guardian live map support. +- AI configuration, voice command configuration, hardware shortcut configuration. +- WalkGuide obstacle detection pipeline integration points. +- Call token and call notification endpoints for Agora-style VoIP flow. ## System Architecture -The study follows a strict three-pillar enterprise structure: +The project follows a feature-first architecture across backend, mobile, and OOAD documents. -**Pillar 1 — OOAD (Object-Oriented Analysis & Design):** Comprehensive modeling using Use Case, Class, Sequence, and ERD diagrams. The codebase strictly implements ≥ 7 GoF Design Patterns (Builder, Singleton, Facade, Repository/Proxy, Observer, Strategy, Chain of Responsibility). +### Backend -**Pillar 2 — Flutter (Mobile Frontend):** Implements Clean Architecture (Domain, Data, Presentation layers) with BLoC for state management. Uses `Dio` with interceptors for secure HTTP communication. Server URL is dynamically configured via `SharedPreferences` on first launch. +Backend follows layered Spring Boot architecture: -**Pillar 3 — Spring Boot (Backend API):** A layered architecture (Controller → Service → Repository) powered by Java 21. Features JWT-based Role-Based Access Control (RBAC), standardized `ApiResponse` envelopes, WebSocket (STOMP) for real-time data, and Firebase Admin SDK for push notifications. +```text +Controller -> Service -> Repository -> Entity -> PostgreSQL +``` ---- +Main backend concerns: + +- `controller/`: REST API endpoints. +- `service/`: application/business logic. +- `repository/`: Spring Data JPA persistence contracts. +- `entity/`: database-mapped entities. +- `dto/request` and `dto/response`: API input/output contracts. +- `security/`: JWT utility, JWT filter, custom user details service. +- `config/`: Security, WebSocket, OpenAPI, Firebase/FCM, seeding. +- `websocket/`: STOMP broadcasting helper. +- `db/migration/`: Flyway schema migrations. + +### Flutter + +Flutter uses a feature-first layout. Several critical features now have domain/data/application/presentation structure, while compatibility wrapper screens remain for the existing app routes. + +Main Flutter concerns: + +- `app/`: app shell, router, dependency injection. +- `core/`: API service/client, services, storage/cache, AI helpers, errors. +- `features/`: auth, server connect, pairing, SOS, notifications, WalkGuide, activity log, navigation, settings, call, guardian dashboard/tools, manual, benchmark. +- `shared/widgets/`: common UI shell and reusable feature page components. + +### OOAD + +The `ooad-docs/` folder contains traceability and diagrams, including: + +- `01_Builder_Pattern.puml` +- `02_Singleton_Pattern.puml` +- `03_Facade_Pattern.puml` +- `04_Repository_Proxy_Pattern.puml` +- `05_Observer_Pattern.puml` +- `06_Strategy_Pattern.puml` +- `07_ChainOfResponsibility_Pattern.puml` +- Use case, sequence, state, ERD, class, and component diagrams under `ooad-docs/diagrams/`. ## Tech Stack -### Backend (Spring Boot) +### Backend -| Library / Tool | Version | Purpose | +| Tool / Library | Current Codebase | Purpose | |---|---|---| -| Java | 21 | Primary language | -| Spring Boot | 3.3.x | Main framework | -| Spring Security | (bundled) | Auth + RBAC | -| Spring Data JPA | (bundled) | ORM | -| Spring WebSocket (STOMP) | (bundled) | Real-time location & notification push | -| PostgreSQL Driver | (bundled) | DB connection — university server `202.46.28.160` | -| Flyway | 10.x | Database schema migration | -| JJWT | 0.11.5 | JWT access + refresh token | -| Firebase Admin SDK | 9.x | FCM push notifications | -| Agora RESTful API | - | Generate Agora RTC token for VoIP | -| Springdoc OpenAPI | 2.3.0 | Swagger UI documentation | -| Lombok | latest | Boilerplate reduction | -| JUnit 5 + Mockito | (bundled) | Unit testing | -| MockMvc + Testcontainers | 1.19.x | Integration testing with real PostgreSQL | -| JaCoCo | 0.8.x | Code coverage (target ≥ 70%) | +| Java | 21 | Backend language | +| Spring Boot | 3.2.5 | Main backend framework | +| Spring Security | Spring Boot managed | JWT auth and RBAC | +| Spring Data JPA | Spring Boot managed | ORM and repositories | +| Spring WebSocket | Spring Boot managed | STOMP realtime channels | +| PostgreSQL Driver | Runtime dependency | University PostgreSQL database | +| Flyway | Core + PostgreSQL module | Database migrations | +| JJWT | 0.11.5 | JWT access token handling | +| Springdoc OpenAPI | 2.3.0 | Swagger/OpenAPI docs | +| Lombok | 1.18.36 | Builders and boilerplate reduction | +| JUnit 5 / Mockito / MockMvc | Test dependencies | Unit and controller testing | +| Testcontainers | 1.19.7 | PostgreSQL-backed integration tests | +| JaCoCo | 0.8.11 | Coverage report | -### Flutter (Mobile) +### Flutter -| Package | Version | Purpose | +| Package | Current Codebase | Purpose | |---|---|---| -| flutter_bloc | 8.x | State management (sole pattern) | -| go_router | 14.x | Navigation + role-based route guards | -| dio | 5.x | HTTP client with interceptors | -| shared_preferences | 2.x | Persist dynamic server URL | -| flutter_secure_storage | 10.x | Secure JWT token storage | -| drift | 2.x | SQLite ORM for offline cache | -| tflite_flutter | 0.10.x | Run YOLOv8n on-device | -| camera | 0.10.x | Camera feed for YOLO inference | -| flutter_tts | 4.x | Text-to-Speech (ID + EN) | -| speech_to_text | 6.x | Always-listening voice commands | -| agora_rtc_engine | 6.x | VoIP call Guardian ↔ User | -| firebase_messaging | 14.x | FCM push notification receiver | -| flutter_map | 6.x | OpenStreetMap (free, no API key) | -| geolocator | 11.x | Real-time GPS | -| stomp_dart_client | 2.x | WebSocket STOMP for live tracking | -| get_it | 7.x | Service locator / dependency injection | -| dartz | 0.10.x | `Either` typed error handling | -| vibration | 1.x | Haptic feedback on obstacle detection | +| flutter_bloc | 8.1.6 | Cubit/BLoC state management | +| go_router | 14.2.7 | App routing | +| dio | 5.4.3+1 | REST client | +| flutter_secure_storage | 9.2.2 | Secure token storage on mobile | +| shared_preferences | 2.3.2 | Server URL and web cache fallback | +| drift / sqlite3 | drift 2.18.0, sqlite3 2.4.7 | Local persistence support | +| sqlite3_flutter_libs | 0.5.24 | Android/iOS SQLite native libs | +| camera | 0.11.0+2 | Camera feed | +| tflite_flutter | 0.12.1 | On-device model inference | +| flutter_tts | 4.0.2 | Text-to-speech | +| speech_to_text | 7.0.0 | Voice recognition support | +| firebase_core / firebase_messaging | 3.3.0 / 15.1.0 | FCM integration | +| flutter_local_notifications | 17.2.1+2 | Foreground/local notification UI | +| flutter_map / latlong2 | 7.0.2 / 0.9.1 | OpenStreetMap UI | +| geolocator | 12.0.0 | GPS/location | +| agora_rtc_engine | 6.3.2 | VoIP engine integration | +| just_audio / record | 0.9.40 / 5.1.2 | Voice note playback/recording | +| get_it | 8.0.2 | Dependency injection | +| dartz | 0.10.1 | Either-style error handling | +| connectivity_plus | 6.0.3 | Offline/online detection | -### External Services +## Runtime Configuration -| Service | Purpose | Cost | -|---|---|---| -| Firebase (FCM) | Push notifications | Free | -| Agora RTC | In-app VoIP audio call | 10,000 min/month free | -| University Server PostgreSQL | DB at `202.46.28.160:2002` | Free (managed by lecturer) | -| OpenStreetMap + OSRM | Map tiles + turn-by-turn routing | Free | -| YOLOv8n (Ultralytics) | Obstacle detection model (.tflite) | Free (open source) | +### Backend ---- +Local/dev config has fallback values so the project can run from an IDE without manually setting every environment variable. + +`application.properties` and `application-dev.yml` currently default to: + +```properties +spring.datasource.url=jdbc:postgresql://202.46.28.160:2002/uas_5803024001 +spring.datasource.username=5803024001 +spring.datasource.password=pw5803024001 +``` + +JWT also has a dev fallback secret. Production config remains strict in `application-prod.yml` and expects environment variables: + +```text +DB_URL +DB_USERNAME +DB_PASSWORD +JWT_SECRET +AGORA_APP_ID +AGORA_APP_CERTIFICATE +``` + +### Flutter + +The Flutter app uses a dynamic server URL. On first launch, use the Server Connect screen and enter a backend URL such as: + +```text +http://202.46.28.160:8080 +``` + +For testing on a physical phone against a backend running on a laptop, do not use `localhost`. Use the laptop LAN IP: + +```text +http://192.168.x.x:8080 +``` ## Implementations -### Design A — User Mode (Visually Impaired) -An accessibility-first interface that opens directly to an active camera view. -- **On-Device AI:** YOLOv8n obstacle detection via TFLite runs locally — zero network latency. Detects 80 COCO object classes, reports direction (LEFT/CENTER/RIGHT) and estimated distance. -- **Voice Commands:** 14 configurable voice commands (e.g., "Start Walkguide", "Call Guardian", "Send SOS", "Where Am I"). Always-listening `SpeechToText` with auto-restart. -- **Hardware Mapping:** Physical volume buttons mapped to critical actions (Vol Up = Call Guardian, Vol Down = Start WalkGuide by default). -- **TTS Feedback:** Queued and immediate Text-to-Speech for obstacle alerts, navigation, and screen announcements. Supports Bahasa Indonesia and English. -- **SOS System:** One-command emergency alert triggers high-priority FCM to Guardian with GPS coordinates. +### User Mode -### Design B — Guardian Mode (Admin/Caregiver) -A command center dashboard for oversight and remote configuration. -- **Real-time Live Map:** Tracks User's GPS location via WebSocket STOMP subscription, rendered on OpenStreetMap with `flutter_map`. Includes geofence circle overlay. -- **Remote AI Configuration:** Adjusts YOLO confidence threshold, alert distance thresholds, max inference FPS, and enabled object labels. -- **Voice Command Management:** Edits trigger phrases and enables/disables any of the 14 voice commands remotely. -- **Notifications:** Sends text messages or voice notes (recorded audio) directly to the User's device. -- **Geofence:** Sets a geographic boundary — backend notifies Guardian via FCM when User exits. +- WalkGuide camera screen and AI pipeline integration points. +- TTS feedback and friendly error handling. +- SOS screen with emergency action flow. +- Notifications screen with read/read-all actions. +- Activity log, navigation, settings, pairing, and manual screens. +- Voice command and shortcut configuration retrieval paths. +- Offline queue/cache support for core app data. ---- +### Guardian Mode + +- Guardian dashboard and tools. +- Guardian live map screen. +- Guardian activity log screen. +- Send notification screen with text and voice note modes. +- AI config screen. +- Voice command and shortcut management screens. +- Geofence/settings screens. +- SOS acknowledge and resolve support. + +### Backend API + +- Auth, pairing, user, guardian, and shared call controllers. +- Service layer for pairing, location, activity, obstacles, notifications, SOS, AI config, voice commands, hardware shortcuts, geofence, user settings, call notifications, and dashboard aggregation. +- WebSocket broadcaster for location/SOS/notification-style real-time updates. +- Flyway-managed PostgreSQL schema. ## Database Schema -Database runs on PostgreSQL at `202.46.28.160:2002`. Schema is managed exclusively by Flyway migrations. `spring.jpa.hibernate.ddl-auto=validate`. +Database: PostgreSQL on the university server. +Schema management: Flyway. +Hibernate mode: `validate`. -| Migration | Table | Status | +Current migrations in `walkguide-backend/demo/src/main/resources/db/migration/`: + +| Migration | File | Status | |---|---|---| -| V1 | `users` | ✅ Exists | -| V2 | Seed data | ✅ Exists | -| V3 | Guardian-User link | ✅ Exists | -| V4 | Add `unique_user_id` column | 🔄 Needs creation | -| V5 | `pairing_relations` | 🔄 Needs creation | -| V6 | `activity_logs` | 🔄 Needs creation | -| V7 | `obstacle_logs` | 🔄 Needs creation | -| V8 | `location_history` | 🔄 Needs creation | -| V9 | `guardian_notifications` | 🔄 Needs creation | -| V10 | `sos_events` | 🔄 Needs creation | -| V11 | `user_settings` | 🔄 Needs creation | -| V12 | `ai_configs` | 🔄 Needs creation | -| V13 | `voice_command_configs` | 🔄 Needs creation | -| V14 | `hardware_shortcuts` | 🔄 Needs creation | -| V15 | `geofence_configs` | 🔄 Needs creation | -| V16 | `refresh_tokens` | 🔄 Needs creation | - ---- +| V1 | `V1__create_users_table.sql` | Present | +| V2 | `V2__seed_users.sql` | Present | +| V3 | `V3__link_guardian_user.sql` | Present | +| V4 | `V4__alter_users_add_columns.sql` | Present | +| V5 | `V5__create_pairing_relations.sql` | Present | +| V6 | `V6__create_activity_logs.sql` | Present | +| V7 | `V7__create_obstacle_logs.sql` | Present | +| V8 | `V8__create_location_history.sql` | Present | +| V9 | `V9__create_guardian_notifications.sql` | Present | +| V10 | `V10__create_sos_events.sql` | Present | +| V11 | `V11__create_user_settings.sql` | Present | +| V12 | `V12__create_ai_configs.sql` | Present | +| V13 | `V13__create_voice_command_configs.sql` | Present | +| V14 | `V14__create_hardware_shortcuts.sql` | Present | +| V15 | `V15__create_geofence_configs.sql` | Present | +| V16 | `V16__create_refresh_tokens.sql` | Present | +| V17 | `V17__add_expiring_pairing_codes.sql` | Present | ## API Endpoints -26 REST endpoints across 5 groups (exam minimum: 10). - -**Auth** — `/api/v1/auth` - -| Method | Path | Auth | Description | -|---|---|---|---| -| GET | `/ping` | None | Health check / server connection test | -| POST | `/register` | None | Register Guardian or User | -| POST | `/login` | None | Login, returns access + refresh token | -| POST | `/refresh` | None | Refresh access token | -| POST | `/logout` | ✅ | Logout, invalidate refresh token | -| PUT | `/fcm-token` | ✅ | Update FCM device token | - -**Pairing** — `/api/v1/shared/pairing` - -| Method | Path | Auth | Description | -|---|---|---|---| -| POST | `/invite` | GUARDIAN | Invite User by 12-char unique ID | -| POST | `/respond` | USER | Accept or reject pairing invite | -| DELETE | `/unpair` | ✅ | Dissolve active pairing | -| GET | `/status` | ✅ | Get current pairing status | - -**Guardian** — `/api/v1/guardian` +### Auth - `/api/v1/auth` | Method | Path | Description | |---|---|---| -| GET | `/dashboard` | Combined home data (user status, recent logs, SOS count) | -| GET | `/user-status` | Full status of paired User | -| GET | `/user-location` | Last known GPS location | +| GET | `/ping` | Server connection check | +| POST | `/register` | Register Guardian or User | +| POST | `/login` | Login | +| POST | `/refresh` | Refresh access token | +| POST | `/logout` | Logout | +| PUT | `/fcm-token` | Update FCM token | + +### Pairing - `/api/v1/shared/pairing` + +| Method | Path | Description | +|---|---|---| +| GET | `/code` | Get current pairing code | +| POST | `/code/regenerate` | Regenerate pairing code | +| POST | `/invite` | Guardian invites User | +| POST | `/respond` | User accepts/rejects invite | +| DELETE | `/unpair` | Remove pairing | +| GET | `/status` | Get current pairing status | + +### Guardian - `/api/v1/guardian` + +| Method | Path | Description | +|---|---|---| +| GET | `/dashboard` | Guardian home data | +| GET | `/user-location` | Last known User location | | GET | `/location-history` | Paginated location history | -| GET | `/activity-logs` | Paginated activity logs of paired User | -| GET | `/obstacle-logs` | Paginated obstacle detection logs | -| POST | `/notifications/send` | Send text or voice note to User | +| GET | `/activity-logs` | Paginated User activity logs | +| GET | `/obstacle-logs` | Paginated obstacle logs | +| POST | `/notifications/send` | Send text or voice note notification | | GET | `/sos-events` | Paginated SOS events | -| PUT | `/sos/{id}/acknowledge` | Acknowledge SOS alert | -| GET/PUT | `/ai-config` | Get or update AI configuration | -| GET/PUT | `/voice-commands` | Get or update voice command configs | -| GET/PUT | `/shortcuts` | Get or update hardware shortcut configs | -| GET/PUT | `/geofence` | Get or update geofence config | +| PUT | `/sos/{id}/acknowledge` | Mark SOS as acknowledged | +| PUT | `/sos/{id}/resolve` | Mark SOS as resolved/handled | +| GET/PUT | `/ai-config` | Get/update AI config | +| GET/PUT | `/voice-commands` | Get/update voice command config | +| GET/PUT | `/shortcuts` | Get/update shortcut config | +| GET/PUT | `/geofence` | Get/update geofence config | +| GET/PUT | `/user-settings` | Get/update paired User settings | -**User** — `/api/v1/user` +### User - `/api/v1/user` | Method | Path | Description | |---|---|---| -| GET | `/profile` | User profile (id, email, displayName, uniqueUserId) | -| GET/PUT | `/settings` | Get or update TTS/haptic settings | -| GET | `/voice-commands` | Get all voice commands (read-only) | -| GET/PUT | `/shortcuts` | Get or capture hardware shortcut assignments | -| GET | `/ai-config` | Get AI config (read-only) | -| POST | `/location` | Send GPS update | -| POST | `/obstacle` | Log detected obstacle | -| POST | `/sos` | Trigger SOS alert | -| GET | `/activity-logs` | Paginated own activity log | -| GET | `/notifications` | Paginated notifications from Guardian | -| GET | `/notifications/unread-count` | Unread notification count | -| PUT | `/notifications/mark-all-read` | Mark all as read | -| PUT | `/notifications/{id}/read` | Mark one as read | -| POST | `/walkguide/start` | Log WalkGuide session start | -| POST | `/walkguide/stop` | Log WalkGuide session stop | +| GET | `/profile` | Current User profile | +| GET/PUT | `/settings` | Get/update User settings | +| GET | `/voice-commands` | Get voice command config | +| GET/PUT | `/shortcuts` | Get/update shortcut config | +| GET | `/ai-config` | Get AI config | +| POST | `/location` | Send location update | +| POST | `/obstacle` | Log obstacle | +| POST | `/sos` | Trigger SOS | +| GET | `/sos-events` | Get own SOS events | +| GET | `/activity-logs` | Get activity logs | +| GET | `/notifications` | Get notifications | +| GET | `/notifications/unread-count` | Get unread count | +| PUT | `/notifications/mark-all-read` | Mark all notifications read | +| PUT | `/notifications/{id}/read` | Mark one notification read | +| POST | `/walkguide/start` | Log WalkGuide start | +| POST | `/walkguide/stop` | Log WalkGuide stop | -**Shared Call** — `/api/v1/shared/call` +### Shared Call - `/api/v1/shared/call` | Method | Path | Description | |---|---|---| -| POST | `/token` | Generate Agora RTC token | -| POST | `/notify` | Send "Incoming Call" FCM to other party | - ---- +| POST | `/token` | Generate call token/channel payload | +| POST | `/notify` | Notify other party of incoming call | +| POST | `/end` | Notify/end call session | ## Design Patterns -7 GoF Design Patterns implemented (exam minimum: 4, minimum 1 per category). +The project documents and maps 7 GoF-related patterns in `ooad-docs/`. -| # | Category | Pattern | Location | +| # | Category | Pattern | Main Location | |---|---|---|---| -| 1 | Creational | **Builder** | `User.java` (`@Builder`), `FcmService` message construction | -| 2 | Creational | **Singleton** | Flutter: `TtsService`, `YoloDetector`, `WebSocketService`, `AgoraService` via GetIt | -| 3 | Structural | **Facade** | Flutter: `VoiceCommandHandler`; Backend: `GuardianDashboardService` | -| 4 | Structural | **Repository (Proxy)** | All `*_repository_impl.dart` — proxy between domain and data sources | -| 5 | Behavioral | **Observer** | BLoC pattern (BLoC = Subject, Widgets = Observers); WebSocket callbacks | -| 6 | Behavioral | **Strategy** | `ObstacleAnalyzer` direction strategy; `ObstacleAlertStrategyService` (backend) | -| 7 | Behavioral | **Chain of Responsibility** | Spring Security filter chain; Dio interceptor chain | +| 1 | Creational | Builder | Backend entities/DTO construction, Lombok builders | +| 2 | Creational | Singleton | Flutter services registered through GetIt | +| 3 | Structural | Facade | Guardian dashboard aggregation and voice command/service coordination | +| 4 | Structural | Repository / Proxy | Spring Data repositories and Flutter repository-style data access | +| 5 | Behavioral | Observer | BLoC/Cubit state listeners and WebSocket callbacks | +| 6 | Behavioral | Strategy | Obstacle analysis / alert behavior mapping documented in OOAD | +| 7 | Behavioral | Chain of Responsibility | Spring Security filter chain and Dio interceptors | ---- +See: -## Metrics +- `ooad-docs/DESIGN_PATTERNS.md` +- `ooad-docs/TRACEABILITY_AUDIT.md` +- `ooad-docs/01_Builder_Pattern.puml` through `ooad-docs/07_ChainOfResponsibility_Pattern.puml` -| Metric | Instrument | Target | -|---|---|---| -| API Throughput | Apache JMeter / k6 | ≥ 100 req/s under load | -| API p95 Latency | Apache JMeter / k6 | < 500ms | -| UI Frame Rate | Flutter DevTools | ≥ 90% frames < 16ms | -| Code Coverage | JaCoCo | ≥ 70% (Service & Controller) | -| APK Size | `flutter build apk --analyze-size` | < 50MB | +## Metrics And Evidence ---- +The repository includes testing and benchmark support, but final scoring evidence should be generated on the target machine/device before submission. + +| Area | Current Support | +|---|---| +| Backend unit tests | JUnit/Mockito tests under `src/test/java` | +| Backend controller/integration tests | MockMvc and Testcontainers setup present | +| Coverage | JaCoCo configured in Maven | +| Load testing | k6 assets/results folder present under backend project | +| Flutter tests | Unit/widget/integration test files present | +| Flutter performance | Benchmark evidence template in `ooad-docs/BENCHMARK_EVIDENCE_TEMPLATE.md` | + +Recommended final evidence: + +- `mvn test` or `mvn verify` output. +- JaCoCo HTML report. +- Testcontainers run on a Docker-enabled machine. +- k6 result at 50 or more concurrent virtual users. +- Flutter physical Android profile evidence: cold start, memory, jank/frame timing, CPU during AI, API latency, APK size. ## Repository Structure -> 🚨 **DISCLAIMER:** This repository is in active development. The directory tree below reflects the intended final architecture. Flutter implementation is currently pending — backend (Spring Boot) is the active development phase. - ```text / -├── walkguide-backend/ # [Spring Boot Engine — ACTIVE] -│ ├── src/main/java/com/walkguide/ -│ │ ├── config/ # SecurityConfig, WebSocketConfig, OpenApiConfig, FcmConfig, DataSeeder -│ │ ├── enums/ # UserRole, PairingStatus, ActivityLogType, VoiceCommandKey, -│ │ │ # HardwareShortcutKey, NotificationType, SosStatus -│ │ ├── entity/ # User, PairingRelation, ActivityLog, ObstacleLog, -│ │ │ # LocationHistory, GuardianNotification, SosEvent, -│ │ │ # UserSettings, AiConfig, VoiceCommandConfig, -│ │ │ # HardwareShortcut, GeofenceConfig, RefreshToken -│ │ ├── repository/ # JPA repositories for all entities -│ │ ├── dto/ -│ │ │ ├── request/ # RegisterRequest, LoginRequest, InviteUserRequest, -│ │ │ │ # LocationUpdateRequest, ObstacleLogRequest, -│ │ │ │ # SendNotificationRequest, SosRequest, -│ │ │ │ # AiConfigUpdateRequest, VoiceCommandUpdateRequest, -│ │ │ │ # HardwareShortcutUpdateRequest, GeofenceConfigRequest, -│ │ │ │ # UserSettingsUpdateRequest, and more -│ │ │ └── response/ # ApiResponse (standard wrapper), AuthDataResponse, -│ │ │ # UserProfileResponse, PairingStatusResponse, -│ │ │ # ActivityLogResponse, ObstacleLogResponse, -│ │ │ # LocationResponse, NotificationResponse, -│ │ │ # SosEventResponse, AgoraTokenResponse, -│ │ │ # AiConfigResponse, VoiceCommandResponse, -│ │ │ # HardwareShortcutResponse, GeofenceResponse -│ │ ├── service/ # AuthService, PairingService, ActivityLogService, -│ │ │ # LocationService, ObstacleLogService, -│ │ │ # NotificationService, SosService, -│ │ │ # AiConfigService, VoiceCommandService, -│ │ │ # HardwareShortcutService, GeofenceService, -│ │ │ # UserSettingsService, FcmService, -│ │ │ # AgoraTokenService, GuardianDashboardService -│ │ ├── controller/ # AuthController, PairingController, -│ │ │ # GuardianController, UserController, CallController -│ │ ├── security/ # JwtUtil, JwtAuthFilter, CustomUserDetailsService -│ │ ├── websocket/ # LocationBroadcaster (STOMP broadcaster) -│ │ └── exception/ # GlobalExceptionHandler, ResourceNotFoundException, -│ │ # UnauthorizedException, PairingException -│ ├── src/main/resources/ -│ │ ├── application.properties # DB config (PostgreSQL 202.46.28.160:2002), Flyway, JWT -│ │ ├── firebase/ -│ │ │ └── google-services-admin.json # FCM service account key (gitignored!) -│ │ └── db/migration/ -│ │ ├── V1__create_users_table.sql ✅ Exists -│ │ ├── V2__seed_users.sql ✅ Exists -│ │ ├── V3__link_guardian_user.sql ✅ Exists -│ │ ├── V4__add_unique_user_id.sql 🔄 Needs creation -│ │ ├── V5__create_pairing_relations.sql 🔄 Needs creation -│ │ ├── V6__create_activity_logs.sql 🔄 Needs creation -│ │ ├── V7__create_obstacle_logs.sql 🔄 Needs creation -│ │ ├── V8__create_location_history.sql 🔄 Needs creation -│ │ ├── V9__create_notifications.sql 🔄 Needs creation -│ │ ├── V10__create_sos_events.sql 🔄 Needs creation -│ │ ├── V11__create_user_settings.sql 🔄 Needs creation -│ │ ├── V12__create_ai_configs.sql 🔄 Needs creation -│ │ ├── V13__create_voice_commands.sql 🔄 Needs creation -│ │ ├── V14__create_hardware_shortcuts.sql 🔄 Needs creation -│ │ ├── V15__create_geofence_configs.sql 🔄 Needs creation -│ │ └── V16__create_refresh_tokens.sql 🔄 Needs creation -│ └── src/test/java/com/walkguide/ -│ ├── service/ # AuthServiceTest, PairingServiceTest, -│ │ # NotificationServiceTest, SosServiceTest, -│ │ # LocationServiceTest, AiConfigServiceTest -│ └── controller/ # AuthControllerTest, GuardianControllerTest, -│ # UserControllerTest (MockMvc + Testcontainers) -│ -├── walkguide-mobile/ # [Flutter Frontend — PENDING] -│ ├── lib/ -│ │ ├── main.dart # Firebase init, GetIt setup, camera init -│ │ ├── app/ -│ │ │ ├── app.dart # MultiBlocProvider + MaterialApp.router -│ │ │ ├── router.dart # GoRouter: /server-connect → /splash → role-based routes -│ │ │ └── injection_container.dart # GetIt singletons & factories -│ │ ├── core/ -│ │ │ ├── constants/ # app_constants.dart (dynamic BASE_URL via SharedPreferences), -│ │ │ │ # route_constants.dart, voice_command_keys.dart -│ │ │ ├── errors/ # failures.dart, exceptions.dart -│ │ │ ├── network/ # api_client.dart (Dio), auth_interceptor.dart, -│ │ │ │ # error_interceptor.dart, network_info.dart -│ │ │ ├── storage/ # secure_storage.dart (JWT), local_database.dart (Drift) -│ │ │ ├── services/ # tts_service.dart, stt_service.dart, -│ │ │ │ # voice_command_handler.dart, -│ │ │ │ # hardware_shortcut_listener.dart, -│ │ │ │ # fcm_service.dart, haptic_service.dart, -│ │ │ │ # websocket_service.dart, agora_service.dart -│ │ │ ├── ai/ # model_loader.dart, yolo_detector.dart, -│ │ │ │ # obstacle_analyzer.dart -│ │ │ ├── theme/ # app_theme.dart, app_colors.dart -│ │ │ └── utils/ # permission_manager.dart, location_service.dart -│ │ ├── features/ -│ │ │ ├── server_connect/ # ServerConnectScreen — FIRST SCREEN on fresh install -│ │ │ │ # BLoC: TestConnection, SaveAndContinue, ChangeServer -│ │ │ ├── auth/ # SplashScreen, LoginScreen, RegisterScreen -│ │ │ │ # Clean Arch: domain / data / presentation -│ │ │ ├── pairing/ # UserPairingScreen (show unique ID, accept invite) -│ │ │ │ # GuardianPairingScreen (invite by 12-char ID) -│ │ │ ├── walk_guide/ # WalkGuideScreen (full-screen camera + YOLO overlay) -│ │ │ │ # WalkGuideBloc: Start → camera stream → detect → TTS -│ │ │ ├── sos/ # SosScreen (large SOS button) -│ │ │ ├── activity_log/ # ActivityLogScreen (paginated, filterable) -│ │ │ ├── notifications/ # NotificationScreen + TTS read-all feature -│ │ │ ├── navigation_mode/ # NavigationModeScreen (OSM map + OSRM routing) -│ │ │ ├── settings/ # UserSettingsScreen (TTS, pairing, account) -│ │ │ ├── call/ # CallScreen + IncomingCallScreen (Agora VoIP) -│ │ │ ├── manual/ # ManualScreen (all voice commands & shortcuts) -│ │ │ └── guardian_dashboard/ # GuardianDashboardScreen, GuardianMapScreen, -│ │ │ # GuardianActivityLogScreen, GuardianSendNotifScreen, -│ │ │ # GuardianAiConfigScreen, GuardianVoiceCmdScreen, -│ │ │ # GuardianShortcutScreen, GuardianGeofenceScreen -│ │ └── shared/widgets/ # AppBottomNav, VoiceCommandListener, -│ │ # TtsAwareScaffold, LoadingOverlay, ErrorWidget -│ ├── assets/ -│ │ ├── models/ -│ │ │ ├── yolov8n.tflite # YOLOv8n converted to TFLite (~6MB) -│ │ │ └── labels.txt # 80 COCO object labels -│ │ └── images/ -│ └── pubspec.yaml -│ -├── ooad-docs/ # [Design Artifacts] -│ ├── diagrams/ # PlantUML / draw.io exports -│ └── Traceability_Matrix.md -│ -└── README.md +|-- walkguide-backend/ +| |-- demo/ +| |-- src/main/java/com/walkguide/ +| | |-- config/ +| | |-- controller/ +| | |-- dto/ +| | |-- entity/ +| | |-- enums/ +| | |-- exception/ +| | |-- repository/ +| | |-- security/ +| | |-- service/ +| | `-- websocket/ +| |-- src/main/resources/ +| | |-- application.properties +| | |-- application-dev.yml +| | |-- application-prod.yml +| | `-- db/migration/V1...V17 +| |-- src/test/java/com/walkguide/ +| |-- k6-tests/ +| `-- pom.xml +| +|-- walkguide-mobile/ +| `-- walkguide_app/ +| |-- lib/ +| | |-- app/ +| | |-- core/ +| | |-- features/ +| | `-- shared/ +| |-- assets/ +| | |-- images/ +| | `-- models/ +| |-- test/ +| |-- integration_test/ +| `-- pubspec.yaml +| +|-- ooad-docs/ +| |-- 01_Builder_Pattern.puml +| |-- 02_Singleton_Pattern.puml +| |-- 03_Facade_Pattern.puml +| |-- 04_Repository_Proxy_Pattern.puml +| |-- 05_Observer_Pattern.puml +| |-- 06_Strategy_Pattern.puml +| |-- 07_ChainOfResponsibility_Pattern.puml +| `-- diagrams/ +| +|-- FULL_FLOW_ARCHITECTURE.md +|-- FINAL_EXAM_GUIDE.md +|-- TODO.md +`-- README.md ``` ---- +## Feature Flows -## Feature Flows (High-Level) +### Flow 1 - Register And Pairing -### Flow 1 — Register & Pairing -Guardian registers → User registers (gets a 12-char `uniqueUserId`) → Guardian enters User's ID → backend sends FCM invite → User accepts in app → backend seeds 14 default voice commands, 5 hardware shortcuts, and AI config for the pair. +Guardian/User registers -> backend creates account and role data -> User receives pairing identity/code -> Guardian sends invite -> User responds -> backend activates pairing and seeds related configs. -### Flow 2 — WalkGuide Active Detection -User says "Start Walkguide" (or presses Volume Down) → camera starts → frames are throttled to max inference FPS → YOLOv8n detects obstacles → direction and distance are analyzed → TTS announces "Caution! Person ahead center. Very close. Please stop." → haptic feedback triggers → obstacle logged to backend → GPS location broadcast via WebSocket to Guardian's live map every 5 seconds. +### Flow 2 - WalkGuide Detection -### Flow 3 — VoIP Call -User says "Call Guardian" → Agora token fetched from backend → FCM "Incoming Call" sent to Guardian → Guardian accepts in `IncomingCallScreen` → both join same Agora RTC channel → audio call connects. +User starts WalkGuide -> camera/AI pipeline runs on device -> obstacle results are analyzed -> TTS/haptic feedback is emitted -> obstacle/location events can be logged to backend. -### Flow 4 — Guardian Sends Notification → User Hears via TTS -Guardian sends text/voice note → backend saves notification + sends FCM → User receives in foreground via WebSocket → `flutter_local_notifications` shown + TTS announced → User says "Read All My Notifications" → TTS reads each message → marked as read. +### Flow 3 - SOS Alert -### Flow 5 — SOS Alert -User says "Send SOS" (or hardware shortcut) → GPS captured → backend saves SOS event → high-priority FCM sent to Guardian → Guardian map opens to User's location → Guardian taps Acknowledge → FCM sent back to User → TTS "Guardian is on the way." +User triggers SOS -> backend creates SOS event -> Guardian receives/loads SOS event -> Guardian taps acknowledge or resolve -> SOS status no longer piles up as unhandled. -### Flow 6 — Geofence -Guardian sets center + radius on map → User's location updates checked against Haversine distance → on exit: FCM to Guardian + `GEOFENCE_EXIT` activity log. +### Flow 4 - Guardian Notification ---- +Guardian opens send notification -> chooses text or voice note mode -> sends payload -> backend stores notification -> User notification screen can display/read/play supported message data. -## Quick Start (WIP) +### Flow 5 - Location And Map -```bash -# 1. Clone the repository -git clone https://github.com/YourGroup/walkguide-final-exam.git +User sends location updates -> backend stores location history -> Guardian map/dashboard reads the latest location and history -> WebSocket support exists for real-time updates. -# 2. Run Backend (Spring Boot) -cd walkguide-backend -./mvnw spring-boot:run -# Flyway auto-migrates V1–V16 on first start -# Swagger UI: http://202.46.28.160:8080/swagger-ui.html -# Health check: GET http://202.46.28.160:8080/api/v1/auth/ping +### Flow 6 - Call -# 3. Run Mobile (Flutter) - Connect to local or university backend -cd ../walkguide-mobile +Caller requests token/channel -> backend returns call token payload -> caller notifies target -> target receives incoming call flow -> call can be ended through shared endpoint. + +## Quick Start + +### Backend + +```powershell +cd "D:\CodeSpace\Final Project Gabungan - Broken Test\walkguide-backend\demo" +.\mvnw.cmd spring-boot:run -Dspring-boot.run.profiles=dev +``` + +Health check: + +```text +http://localhost:8080/api/v1/auth/ping +``` + +Swagger UI: + +```text +http://localhost:8080/swagger-ui.html +``` + +The local/dev profile has fallback DB and JWT values. If you want to override them: + +```powershell +$env:DB_URL="jdbc:postgresql://202.46.28.160:2002/uas_5803024001" +$env:DB_USERNAME="5803024001" +$env:DB_PASSWORD="pw5803024001" +$env:JWT_SECRET="your-base64-secret" +``` + +### Flutter + +```powershell +cd "D:\CodeSpace\Final Project Gabungan - Broken Test\walkguide-mobile\walkguide_app" flutter pub get flutter run -# On first launch: enter server URL in ServerConnectScreen -# Example: http://202.46.28.160:8080 ``` ---- +For Chrome/web debug: + +```powershell +flutter run -d chrome -t lib/main.dart +``` + +For Android APK: + +```powershell +flutter build apk --release +``` + +On a physical phone, the server URL must be reachable by the phone. Use the university server URL or your laptop LAN IP, not `localhost`. ## Results -> ⏳ **Work In Progress:** Results are populated as benchmarking phases are completed. +Final benchmark values should be filled from real test runs before submission. -| Metric | Baseline | Final Optimized | Status | -|---|---|---|---| -| Cold Start Time | *Pending* | *Pending* | ⏳ | -| Memory Leak (10 Navs) | *Pending* | *Pending* | ⏳ | -| API Error Rate | *Pending* | *Pending* | ⏳ | -| YOLO Inference Latency (ms) | *Pending* | *Pending* | ⏳ | -| API p95 Latency | *Pending* | *Pending* | ⏳ | -| JaCoCo Coverage | *Pending* | *Pending* | ⏳ | - ---- +| Metric | Evidence Location / Tool | Current README Status | +|---|---|---| +| Backend tests | Maven/JUnit output | To be generated | +| Backend coverage | JaCoCo report | To be generated | +| Backend load | k6 results | Assets present, final run needed | +| Flutter tests | `flutter test` / integration tests | To be generated | +| Flutter performance | Physical Android profile evidence | To be generated | +| APK size | `flutter build apk --analyze-size` | To be generated | ## Weekly Progress -| Week | Target | Status | +| Week | Target | Current Status | |---|---|---| -| 1 | Topic proposal, Use Case definitions, Repo setup | ✅ Done | -| 2–3 | OOAD diagrams, OpenAPI YAML drafted | 🔄 In Progress | -| 4 | Spring Boot: Auth, Pairing, Entity, Migration V4–V16 | 🔄 In Progress | -| 5 | Spring Boot: Location, SOS, Notification, WS, FCM, Agora | ⏳ Pending | -| 6 | Spring Boot: Unit + Integration tests, JaCoCo ≥ 70% | ⏳ Pending | -| 7 | Flutter: ServerConnect, Auth, WalkGuide + YOLO pipeline | ⏳ Pending | -| 8 | Flutter: Guardian dashboard, Call, SOS, Notifications | ⏳ Pending | -| 9 | Feature freeze, integration testing, benchmark on device | ⏳ Pending | -| 10 | Final benchmarks, Report writing, Demo Video | ⏳ Pending | - ---- +| 1 | Topic proposal, use case definitions, repo setup | Done | +| 2-3 | OOAD diagrams and traceability | Done, docs present in `ooad-docs/` | +| 4 | Spring Boot auth, pairing, entities, migrations | Implemented | +| 5 | Location, SOS, notification, WebSocket, FCM, call support | Implemented with demo/service integrations | +| 6 | Backend unit/integration testing and coverage setup | Implemented, final run evidence needed | +| 7 | Flutter server connect, auth, WalkGuide/YOLO support | Implemented | +| 8 | Guardian dashboard, SOS, notifications, voice notes, settings | Implemented | +| 9 | Integration testing and benchmark evidence | Needs final evidence run | +| 10 | Report, demo video, final submission polish | In progress | ## Core Objectives -**O₁ (Performance):** The dual-platform architecture must maintain a 60fps UI frame rate while handling concurrent backend requests without exceeding 500ms latency. YOLO inference runs on a separate isolate to avoid blocking the UI thread. +O1 Performance: Keep obstacle detection local/on-device where possible and use backend for persistence, pairing, configuration, and real-time coordination. -**O₂ (Accessibility):** The User mode must require zero visual interaction post-login, relying entirely on haptics, voice commands, and physical button mapping. All screen transitions are announced via TTS. +O2 Accessibility: Support voice/TTS, haptic feedback, large touch targets, and hardware shortcut flows for visually impaired users. -**O₃ (Traceability):** Every major feature implementation must be directly traceable back to the pre-development OOAD artifacts and GoF design patterns. +O3 Traceability: Keep implementation aligned with `FULL_FLOW_ARCHITECTURE.md`, `FINAL_EXAM_GUIDE.md`, and the PUML diagrams in `ooad-docs/`. -**O₄ (Configurability):** All AI sensitivity settings, voice command phrases, and hardware shortcuts are remotely configurable by the Guardian without requiring an app update. - ---- +O4 Configurability: Let Guardian configure AI sensitivity, voice commands, hardware shortcuts, geofence, and User settings through dashboard flows. ## License Distributed under the [MIT License](LICENSE). ---- - -*Final Exam: Integrated Mobile Application Project*
-*Flutter × Spring Boot × Object-Oriented Analysis and Design* +Final Exam: Integrated Mobile Application Project +Flutter x Spring Boot x Object-Oriented Analysis and Design