Update README to match current codebase

This commit is contained in:
Wowieee4 2026-05-27 22:19:45 +07:00
parent a629357e8c
commit c6d1e01023

796
README.md
View File

@ -1,489 +1,523 @@
<div align="center">
> ⚠️ **WORK IN PROGRESS (WIP)** ⚠️
> *This repository is currently under active development. All code, data, and structures are subject to continuous changes.*
<br>
<img src="assets/gambar.png" alt="WalkGuide Banner" width="720"/>
# WalkGuide: AI-Powered Navigation
**Integrated Mobile Application Project**
*Flutter Mobile Frontend × Spring Boot Backend × OOAD*
Integrated Mobile Application Project
Flutter Mobile Frontend x Spring Boot Backend x OOAD
### Group Members
WalkGuide is an accessibility-focused mobile system for visually impaired users and their guardians. The User app provides camera-based obstacle awareness, voice/TTS interaction, SOS, notifications, navigation, and call flows. The Guardian app provides monitoring, live location, SOS handling, remote configuration, notification sending, voice notes, and pairing management.
## Group Members
| Name | NIM | Responsibility |
|------|-----|---------------|
| Bambang Herlambang | 5803024019 | - |
| Jap Robertus | 5803024004 | - |
| Evan William | 5803024001 | Backend Engineer (Spring Boot API & Flutter) |
|---|---:|---|
| Bambang Herlambang | 5803024019 | Mobile feature support, documentation, testing |
| Jap Robertus | 5803024004 | Mobile feature support, documentation, testing |
| Evan William | 5803024001 | Backend API, Flutter integration, architecture alignment |
[![Flutter](https://img.shields.io/badge/Flutter-Clean_Architecture-02569B?style=flat-square&logo=flutter&logoColor=white)](https://flutter.dev/)
[![Spring Boot](https://img.shields.io/badge/Spring_Boot-REST_API-6DB33F?style=flat-square&logo=spring&logoColor=white)](https://spring.io/projects/spring-boot)
[![PostgreSQL](https://img.shields.io/badge/PostgreSQL-Database-4169E1?style=flat-square&logo=postgresql&logoColor=white)](https://www.postgresql.org/)
[![Firebase](https://img.shields.io/badge/Firebase-FCM-FFCA28?style=flat-square&logo=firebase&logoColor=black)](https://firebase.google.com/)
[![Agora](https://img.shields.io/badge/Agora-VoIP-099DFD?style=flat-square)](https://www.agora.io/)
[![License](https://img.shields.io/badge/License-MIT-22c55e?style=flat-square)](LICENSE)
## Project Status
![Last Updated](https://img.shields.io/badge/Last_Updated-WIP-blue?style=for-the-badge)
![Author](https://img.shields.io/badge/By-evan--william-brightgreen?style=for-the-badge)
This repository is no longer a backend-only skeleton. The current codebase contains:
[**System Architecture**](#system-architecture) · [**Tech Stack**](#tech-stack) · [**Implementations**](#implementations) · [**API Endpoints**](#api-endpoints) · [**Design Patterns**](#design-patterns) · [**Results**](#results) · [**Weekly Progress**](#weekly-progress)
- Spring Boot backend with JWT/RBAC, Flyway migrations V1-V17, PostgreSQL, OpenAPI, WebSocket, FCM service hooks, Agora token/call notification flow, SOS acknowledge/resolve flow, pairing code flow, service/controller tests, Testcontainers setup, and k6 assets.
- Flutter app with server connection screen, auth, role-based routing, User screens, Guardian screens, feature page UI shell, friendly error handling, offline queue/cache layer, voice note UI support, SOS handling UI, WalkGuide/YOLO support files, and Android/mobile-first dependencies.
- OOAD documentation in `ooad-docs/`, including the 7 GoF design pattern PUML diagrams and traceability documentation.
</div>
Primary demo target: Android APK connected to the Spring Boot backend.
Chrome/web can be used for UI/debug flows, but camera, native AI, SQLite FFI, and mobile permissions are Android-first.
---
## Overview
## Overview — WalkGuide System
Core objective: build an accessible navigation assistant that can help visually impaired users move more safely while allowing a Guardian to monitor, configure, and respond to events in real time.
**Core Objective:** How can we build an ultra-low latency, accessible navigation system for visually impaired users while providing real-time oversight for their guardians?
Important flows implemented or represented in the codebase:
This project implements a dual-interface mobile application. The system relies on **On-Device AI (TFLite / YOLOv8n)** to eliminate network latency during obstacle detection, paired with a robust **Spring Boot backend** for secure authentication, guardian-user pairing, real-time location tracking via WebSocket, and push notifications via Firebase FCM.
**Deployment:** Backend is deployed on a university server at `202.46.28.160`. The Flutter APK uses a dynamic server URL — no hardcoded addresses — allowing multi-device testing without rebuilding.
---
- Register/Login with Guardian and User roles.
- Pairing by generated pairing code / user identity flow.
- Guardian dashboard and tools screens.
- User SOS trigger, Guardian acknowledge and resolve.
- Guardian text and voice-note style notification sending.
- Notification read/read-all handling.
- Location update and Guardian live map support.
- AI configuration, voice command configuration, hardware shortcut configuration.
- WalkGuide obstacle detection pipeline integration points.
- Call token and call notification endpoints for Agora-style VoIP flow.
## System Architecture
The study follows a strict three-pillar enterprise structure:
The project follows a feature-first architecture across backend, mobile, and OOAD documents.
**Pillar 1 — OOAD (Object-Oriented Analysis & Design):** Comprehensive modeling using Use Case, Class, Sequence, and ERD diagrams. The codebase strictly implements ≥ 7 GoF Design Patterns (Builder, Singleton, Facade, Repository/Proxy, Observer, Strategy, Chain of Responsibility).
### Backend
**Pillar 2 — Flutter (Mobile Frontend):** Implements Clean Architecture (Domain, Data, Presentation layers) with BLoC for state management. Uses `Dio` with interceptors for secure HTTP communication. Server URL is dynamically configured via `SharedPreferences` on first launch.
Backend follows layered Spring Boot architecture:
**Pillar 3 — Spring Boot (Backend API):** A layered architecture (Controller → Service → Repository) powered by Java 21. Features JWT-based Role-Based Access Control (RBAC), standardized `ApiResponse` envelopes, WebSocket (STOMP) for real-time data, and Firebase Admin SDK for push notifications.
```text
Controller -> Service -> Repository -> Entity -> PostgreSQL
```
---
Main backend concerns:
- `controller/`: REST API endpoints.
- `service/`: application/business logic.
- `repository/`: Spring Data JPA persistence contracts.
- `entity/`: database-mapped entities.
- `dto/request` and `dto/response`: API input/output contracts.
- `security/`: JWT utility, JWT filter, custom user details service.
- `config/`: Security, WebSocket, OpenAPI, Firebase/FCM, seeding.
- `websocket/`: STOMP broadcasting helper.
- `db/migration/`: Flyway schema migrations.
### Flutter
Flutter uses a feature-first layout. Several critical features now have domain/data/application/presentation structure, while compatibility wrapper screens remain for the existing app routes.
Main Flutter concerns:
- `app/`: app shell, router, dependency injection.
- `core/`: API service/client, services, storage/cache, AI helpers, errors.
- `features/`: auth, server connect, pairing, SOS, notifications, WalkGuide, activity log, navigation, settings, call, guardian dashboard/tools, manual, benchmark.
- `shared/widgets/`: common UI shell and reusable feature page components.
### OOAD
The `ooad-docs/` folder contains traceability and diagrams, including:
- `01_Builder_Pattern.puml`
- `02_Singleton_Pattern.puml`
- `03_Facade_Pattern.puml`
- `04_Repository_Proxy_Pattern.puml`
- `05_Observer_Pattern.puml`
- `06_Strategy_Pattern.puml`
- `07_ChainOfResponsibility_Pattern.puml`
- Use case, sequence, state, ERD, class, and component diagrams under `ooad-docs/diagrams/`.
## Tech Stack
### Backend (Spring Boot)
### Backend
| Library / Tool | Version | Purpose |
| Tool / Library | Current Codebase | Purpose |
|---|---|---|
| Java | 21 | Primary language |
| Spring Boot | 3.3.x | Main framework |
| Spring Security | (bundled) | Auth + RBAC |
| Spring Data JPA | (bundled) | ORM |
| Spring WebSocket (STOMP) | (bundled) | Real-time location & notification push |
| PostgreSQL Driver | (bundled) | DB connection — university server `202.46.28.160` |
| Flyway | 10.x | Database schema migration |
| JJWT | 0.11.5 | JWT access + refresh token |
| Firebase Admin SDK | 9.x | FCM push notifications |
| Agora RESTful API | - | Generate Agora RTC token for VoIP |
| Springdoc OpenAPI | 2.3.0 | Swagger UI documentation |
| Lombok | latest | Boilerplate reduction |
| JUnit 5 + Mockito | (bundled) | Unit testing |
| MockMvc + Testcontainers | 1.19.x | Integration testing with real PostgreSQL |
| JaCoCo | 0.8.x | Code coverage (target ≥ 70%) |
| Java | 21 | Backend language |
| Spring Boot | 3.2.5 | Main backend framework |
| Spring Security | Spring Boot managed | JWT auth and RBAC |
| Spring Data JPA | Spring Boot managed | ORM and repositories |
| Spring WebSocket | Spring Boot managed | STOMP realtime channels |
| PostgreSQL Driver | Runtime dependency | University PostgreSQL database |
| Flyway | Core + PostgreSQL module | Database migrations |
| JJWT | 0.11.5 | JWT access token handling |
| Springdoc OpenAPI | 2.3.0 | Swagger/OpenAPI docs |
| Lombok | 1.18.36 | Builders and boilerplate reduction |
| JUnit 5 / Mockito / MockMvc | Test dependencies | Unit and controller testing |
| Testcontainers | 1.19.7 | PostgreSQL-backed integration tests |
| JaCoCo | 0.8.11 | Coverage report |
### Flutter (Mobile)
### Flutter
| Package | Version | Purpose |
| Package | Current Codebase | Purpose |
|---|---|---|
| flutter_bloc | 8.x | State management (sole pattern) |
| go_router | 14.x | Navigation + role-based route guards |
| dio | 5.x | HTTP client with interceptors |
| shared_preferences | 2.x | Persist dynamic server URL |
| flutter_secure_storage | 10.x | Secure JWT token storage |
| drift | 2.x | SQLite ORM for offline cache |
| tflite_flutter | 0.10.x | Run YOLOv8n on-device |
| camera | 0.10.x | Camera feed for YOLO inference |
| flutter_tts | 4.x | Text-to-Speech (ID + EN) |
| speech_to_text | 6.x | Always-listening voice commands |
| agora_rtc_engine | 6.x | VoIP call Guardian ↔ User |
| firebase_messaging | 14.x | FCM push notification receiver |
| flutter_map | 6.x | OpenStreetMap (free, no API key) |
| geolocator | 11.x | Real-time GPS |
| stomp_dart_client | 2.x | WebSocket STOMP for live tracking |
| get_it | 7.x | Service locator / dependency injection |
| dartz | 0.10.x | `Either<Failure, Data>` typed error handling |
| vibration | 1.x | Haptic feedback on obstacle detection |
| flutter_bloc | 8.1.6 | Cubit/BLoC state management |
| go_router | 14.2.7 | App routing |
| dio | 5.4.3+1 | REST client |
| flutter_secure_storage | 9.2.2 | Secure token storage on mobile |
| shared_preferences | 2.3.2 | Server URL and web cache fallback |
| drift / sqlite3 | drift 2.18.0, sqlite3 2.4.7 | Local persistence support |
| sqlite3_flutter_libs | 0.5.24 | Android/iOS SQLite native libs |
| camera | 0.11.0+2 | Camera feed |
| tflite_flutter | 0.12.1 | On-device model inference |
| flutter_tts | 4.0.2 | Text-to-speech |
| speech_to_text | 7.0.0 | Voice recognition support |
| firebase_core / firebase_messaging | 3.3.0 / 15.1.0 | FCM integration |
| flutter_local_notifications | 17.2.1+2 | Foreground/local notification UI |
| flutter_map / latlong2 | 7.0.2 / 0.9.1 | OpenStreetMap UI |
| geolocator | 12.0.0 | GPS/location |
| agora_rtc_engine | 6.3.2 | VoIP engine integration |
| just_audio / record | 0.9.40 / 5.1.2 | Voice note playback/recording |
| get_it | 8.0.2 | Dependency injection |
| dartz | 0.10.1 | Either-style error handling |
| connectivity_plus | 6.0.3 | Offline/online detection |
### External Services
## Runtime Configuration
| Service | Purpose | Cost |
|---|---|---|
| Firebase (FCM) | Push notifications | Free |
| Agora RTC | In-app VoIP audio call | 10,000 min/month free |
| University Server PostgreSQL | DB at `202.46.28.160:2002` | Free (managed by lecturer) |
| OpenStreetMap + OSRM | Map tiles + turn-by-turn routing | Free |
| YOLOv8n (Ultralytics) | Obstacle detection model (.tflite) | Free (open source) |
### Backend
---
Local/dev config has fallback values so the project can run from an IDE without manually setting every environment variable.
`application.properties` and `application-dev.yml` currently default to:
```properties
spring.datasource.url=jdbc:postgresql://202.46.28.160:2002/uas_5803024001
spring.datasource.username=5803024001
spring.datasource.password=pw5803024001
```
JWT also has a dev fallback secret. Production config remains strict in `application-prod.yml` and expects environment variables:
```text
DB_URL
DB_USERNAME
DB_PASSWORD
JWT_SECRET
AGORA_APP_ID
AGORA_APP_CERTIFICATE
```
### Flutter
The Flutter app uses a dynamic server URL. On first launch, use the Server Connect screen and enter a backend URL such as:
```text
http://202.46.28.160:8080
```
For testing on a physical phone against a backend running on a laptop, do not use `localhost`. Use the laptop LAN IP:
```text
http://192.168.x.x:8080
```
## Implementations
### Design A — User Mode (Visually Impaired)
An accessibility-first interface that opens directly to an active camera view.
- **On-Device AI:** YOLOv8n obstacle detection via TFLite runs locally — zero network latency. Detects 80 COCO object classes, reports direction (LEFT/CENTER/RIGHT) and estimated distance.
- **Voice Commands:** 14 configurable voice commands (e.g., "Start Walkguide", "Call Guardian", "Send SOS", "Where Am I"). Always-listening `SpeechToText` with auto-restart.
- **Hardware Mapping:** Physical volume buttons mapped to critical actions (Vol Up = Call Guardian, Vol Down = Start WalkGuide by default).
- **TTS Feedback:** Queued and immediate Text-to-Speech for obstacle alerts, navigation, and screen announcements. Supports Bahasa Indonesia and English.
- **SOS System:** One-command emergency alert triggers high-priority FCM to Guardian with GPS coordinates.
### User Mode
### Design B — Guardian Mode (Admin/Caregiver)
A command center dashboard for oversight and remote configuration.
- **Real-time Live Map:** Tracks User's GPS location via WebSocket STOMP subscription, rendered on OpenStreetMap with `flutter_map`. Includes geofence circle overlay.
- **Remote AI Configuration:** Adjusts YOLO confidence threshold, alert distance thresholds, max inference FPS, and enabled object labels.
- **Voice Command Management:** Edits trigger phrases and enables/disables any of the 14 voice commands remotely.
- **Notifications:** Sends text messages or voice notes (recorded audio) directly to the User's device.
- **Geofence:** Sets a geographic boundary — backend notifies Guardian via FCM when User exits.
- WalkGuide camera screen and AI pipeline integration points.
- TTS feedback and friendly error handling.
- SOS screen with emergency action flow.
- Notifications screen with read/read-all actions.
- Activity log, navigation, settings, pairing, and manual screens.
- Voice command and shortcut configuration retrieval paths.
- Offline queue/cache support for core app data.
---
### Guardian Mode
- Guardian dashboard and tools.
- Guardian live map screen.
- Guardian activity log screen.
- Send notification screen with text and voice note modes.
- AI config screen.
- Voice command and shortcut management screens.
- Geofence/settings screens.
- SOS acknowledge and resolve support.
### Backend API
- Auth, pairing, user, guardian, and shared call controllers.
- Service layer for pairing, location, activity, obstacles, notifications, SOS, AI config, voice commands, hardware shortcuts, geofence, user settings, call notifications, and dashboard aggregation.
- WebSocket broadcaster for location/SOS/notification-style real-time updates.
- Flyway-managed PostgreSQL schema.
## Database Schema
Database runs on PostgreSQL at `202.46.28.160:2002`. Schema is managed exclusively by Flyway migrations. `spring.jpa.hibernate.ddl-auto=validate`.
Database: PostgreSQL on the university server.
Schema management: Flyway.
Hibernate mode: `validate`.
| Migration | Table | Status |
Current migrations in `walkguide-backend/demo/src/main/resources/db/migration/`:
| Migration | File | Status |
|---|---|---|
| V1 | `users` | ✅ Exists |
| V2 | Seed data | ✅ Exists |
| V3 | Guardian-User link | ✅ Exists |
| V4 | Add `unique_user_id` column | 🔄 Needs creation |
| V5 | `pairing_relations` | 🔄 Needs creation |
| V6 | `activity_logs` | 🔄 Needs creation |
| V7 | `obstacle_logs` | 🔄 Needs creation |
| V8 | `location_history` | 🔄 Needs creation |
| V9 | `guardian_notifications` | 🔄 Needs creation |
| V10 | `sos_events` | 🔄 Needs creation |
| V11 | `user_settings` | 🔄 Needs creation |
| V12 | `ai_configs` | 🔄 Needs creation |
| V13 | `voice_command_configs` | 🔄 Needs creation |
| V14 | `hardware_shortcuts` | 🔄 Needs creation |
| V15 | `geofence_configs` | 🔄 Needs creation |
| V16 | `refresh_tokens` | 🔄 Needs creation |
---
| V1 | `V1__create_users_table.sql` | Present |
| V2 | `V2__seed_users.sql` | Present |
| V3 | `V3__link_guardian_user.sql` | Present |
| V4 | `V4__alter_users_add_columns.sql` | Present |
| V5 | `V5__create_pairing_relations.sql` | Present |
| V6 | `V6__create_activity_logs.sql` | Present |
| V7 | `V7__create_obstacle_logs.sql` | Present |
| V8 | `V8__create_location_history.sql` | Present |
| V9 | `V9__create_guardian_notifications.sql` | Present |
| V10 | `V10__create_sos_events.sql` | Present |
| V11 | `V11__create_user_settings.sql` | Present |
| V12 | `V12__create_ai_configs.sql` | Present |
| V13 | `V13__create_voice_command_configs.sql` | Present |
| V14 | `V14__create_hardware_shortcuts.sql` | Present |
| V15 | `V15__create_geofence_configs.sql` | Present |
| V16 | `V16__create_refresh_tokens.sql` | Present |
| V17 | `V17__add_expiring_pairing_codes.sql` | Present |
## API Endpoints
26 REST endpoints across 5 groups (exam minimum: 10).
**Auth** — `/api/v1/auth`
| Method | Path | Auth | Description |
|---|---|---|---|
| GET | `/ping` | None | Health check / server connection test |
| POST | `/register` | None | Register Guardian or User |
| POST | `/login` | None | Login, returns access + refresh token |
| POST | `/refresh` | None | Refresh access token |
| POST | `/logout` | ✅ | Logout, invalidate refresh token |
| PUT | `/fcm-token` | ✅ | Update FCM device token |
**Pairing** — `/api/v1/shared/pairing`
| Method | Path | Auth | Description |
|---|---|---|---|
| POST | `/invite` | GUARDIAN | Invite User by 12-char unique ID |
| POST | `/respond` | USER | Accept or reject pairing invite |
| DELETE | `/unpair` | ✅ | Dissolve active pairing |
| GET | `/status` | ✅ | Get current pairing status |
**Guardian** — `/api/v1/guardian`
### Auth - `/api/v1/auth`
| Method | Path | Description |
|---|---|---|
| GET | `/dashboard` | Combined home data (user status, recent logs, SOS count) |
| GET | `/user-status` | Full status of paired User |
| GET | `/user-location` | Last known GPS location |
| GET | `/ping` | Server connection check |
| POST | `/register` | Register Guardian or User |
| POST | `/login` | Login |
| POST | `/refresh` | Refresh access token |
| POST | `/logout` | Logout |
| PUT | `/fcm-token` | Update FCM token |
### Pairing - `/api/v1/shared/pairing`
| Method | Path | Description |
|---|---|---|
| GET | `/code` | Get current pairing code |
| POST | `/code/regenerate` | Regenerate pairing code |
| POST | `/invite` | Guardian invites User |
| POST | `/respond` | User accepts/rejects invite |
| DELETE | `/unpair` | Remove pairing |
| GET | `/status` | Get current pairing status |
### Guardian - `/api/v1/guardian`
| Method | Path | Description |
|---|---|---|
| GET | `/dashboard` | Guardian home data |
| GET | `/user-location` | Last known User location |
| GET | `/location-history` | Paginated location history |
| GET | `/activity-logs` | Paginated activity logs of paired User |
| GET | `/obstacle-logs` | Paginated obstacle detection logs |
| POST | `/notifications/send` | Send text or voice note to User |
| GET | `/activity-logs` | Paginated User activity logs |
| GET | `/obstacle-logs` | Paginated obstacle logs |
| POST | `/notifications/send` | Send text or voice note notification |
| GET | `/sos-events` | Paginated SOS events |
| PUT | `/sos/{id}/acknowledge` | Acknowledge SOS alert |
| GET/PUT | `/ai-config` | Get or update AI configuration |
| GET/PUT | `/voice-commands` | Get or update voice command configs |
| GET/PUT | `/shortcuts` | Get or update hardware shortcut configs |
| GET/PUT | `/geofence` | Get or update geofence config |
| PUT | `/sos/{id}/acknowledge` | Mark SOS as acknowledged |
| PUT | `/sos/{id}/resolve` | Mark SOS as resolved/handled |
| GET/PUT | `/ai-config` | Get/update AI config |
| GET/PUT | `/voice-commands` | Get/update voice command config |
| GET/PUT | `/shortcuts` | Get/update shortcut config |
| GET/PUT | `/geofence` | Get/update geofence config |
| GET/PUT | `/user-settings` | Get/update paired User settings |
**User** — `/api/v1/user`
### User - `/api/v1/user`
| Method | Path | Description |
|---|---|---|
| GET | `/profile` | User profile (id, email, displayName, uniqueUserId) |
| GET/PUT | `/settings` | Get or update TTS/haptic settings |
| GET | `/voice-commands` | Get all voice commands (read-only) |
| GET/PUT | `/shortcuts` | Get or capture hardware shortcut assignments |
| GET | `/ai-config` | Get AI config (read-only) |
| POST | `/location` | Send GPS update |
| POST | `/obstacle` | Log detected obstacle |
| POST | `/sos` | Trigger SOS alert |
| GET | `/activity-logs` | Paginated own activity log |
| GET | `/notifications` | Paginated notifications from Guardian |
| GET | `/notifications/unread-count` | Unread notification count |
| PUT | `/notifications/mark-all-read` | Mark all as read |
| PUT | `/notifications/{id}/read` | Mark one as read |
| POST | `/walkguide/start` | Log WalkGuide session start |
| POST | `/walkguide/stop` | Log WalkGuide session stop |
| GET | `/profile` | Current User profile |
| GET/PUT | `/settings` | Get/update User settings |
| GET | `/voice-commands` | Get voice command config |
| GET/PUT | `/shortcuts` | Get/update shortcut config |
| GET | `/ai-config` | Get AI config |
| POST | `/location` | Send location update |
| POST | `/obstacle` | Log obstacle |
| POST | `/sos` | Trigger SOS |
| GET | `/sos-events` | Get own SOS events |
| GET | `/activity-logs` | Get activity logs |
| GET | `/notifications` | Get notifications |
| GET | `/notifications/unread-count` | Get unread count |
| PUT | `/notifications/mark-all-read` | Mark all notifications read |
| PUT | `/notifications/{id}/read` | Mark one notification read |
| POST | `/walkguide/start` | Log WalkGuide start |
| POST | `/walkguide/stop` | Log WalkGuide stop |
**Shared Call** — `/api/v1/shared/call`
### Shared Call - `/api/v1/shared/call`
| Method | Path | Description |
|---|---|---|
| POST | `/token` | Generate Agora RTC token |
| POST | `/notify` | Send "Incoming Call" FCM to other party |
---
| POST | `/token` | Generate call token/channel payload |
| POST | `/notify` | Notify other party of incoming call |
| POST | `/end` | Notify/end call session |
## Design Patterns
7 GoF Design Patterns implemented (exam minimum: 4, minimum 1 per category).
The project documents and maps 7 GoF-related patterns in `ooad-docs/`.
| # | Category | Pattern | Location |
| # | Category | Pattern | Main Location |
|---|---|---|---|
| 1 | Creational | **Builder** | `User.java` (`@Builder`), `FcmService` message construction |
| 2 | Creational | **Singleton** | Flutter: `TtsService`, `YoloDetector`, `WebSocketService`, `AgoraService` via GetIt |
| 3 | Structural | **Facade** | Flutter: `VoiceCommandHandler`; Backend: `GuardianDashboardService` |
| 4 | Structural | **Repository (Proxy)** | All `*_repository_impl.dart` — proxy between domain and data sources |
| 5 | Behavioral | **Observer** | BLoC pattern (BLoC = Subject, Widgets = Observers); WebSocket callbacks |
| 6 | Behavioral | **Strategy** | `ObstacleAnalyzer` direction strategy; `ObstacleAlertStrategyService` (backend) |
| 7 | Behavioral | **Chain of Responsibility** | Spring Security filter chain; Dio interceptor chain |
| 1 | Creational | Builder | Backend entities/DTO construction, Lombok builders |
| 2 | Creational | Singleton | Flutter services registered through GetIt |
| 3 | Structural | Facade | Guardian dashboard aggregation and voice command/service coordination |
| 4 | Structural | Repository / Proxy | Spring Data repositories and Flutter repository-style data access |
| 5 | Behavioral | Observer | BLoC/Cubit state listeners and WebSocket callbacks |
| 6 | Behavioral | Strategy | Obstacle analysis / alert behavior mapping documented in OOAD |
| 7 | Behavioral | Chain of Responsibility | Spring Security filter chain and Dio interceptors |
---
See:
## Metrics
- `ooad-docs/DESIGN_PATTERNS.md`
- `ooad-docs/TRACEABILITY_AUDIT.md`
- `ooad-docs/01_Builder_Pattern.puml` through `ooad-docs/07_ChainOfResponsibility_Pattern.puml`
| Metric | Instrument | Target |
|---|---|---|
| API Throughput | Apache JMeter / k6 | ≥ 100 req/s under load |
| API p95 Latency | Apache JMeter / k6 | < 500ms |
| UI Frame Rate | Flutter DevTools | ≥ 90% frames < 16ms |
| Code Coverage | JaCoCo | ≥ 70% (Service & Controller) |
| APK Size | `flutter build apk --analyze-size` | < 50MB |
## Metrics And Evidence
---
The repository includes testing and benchmark support, but final scoring evidence should be generated on the target machine/device before submission.
| Area | Current Support |
|---|---|
| Backend unit tests | JUnit/Mockito tests under `src/test/java` |
| Backend controller/integration tests | MockMvc and Testcontainers setup present |
| Coverage | JaCoCo configured in Maven |
| Load testing | k6 assets/results folder present under backend project |
| Flutter tests | Unit/widget/integration test files present |
| Flutter performance | Benchmark evidence template in `ooad-docs/BENCHMARK_EVIDENCE_TEMPLATE.md` |
Recommended final evidence:
- `mvn test` or `mvn verify` output.
- JaCoCo HTML report.
- Testcontainers run on a Docker-enabled machine.
- k6 result at 50 or more concurrent virtual users.
- Flutter physical Android profile evidence: cold start, memory, jank/frame timing, CPU during AI, API latency, APK size.
## Repository Structure
> 🚨 **DISCLAIMER:** This repository is in active development. The directory tree below reflects the intended final architecture. Flutter implementation is currently pending — backend (Spring Boot) is the active development phase.
```text
/
├── walkguide-backend/ # [Spring Boot Engine — ACTIVE]
│ ├── src/main/java/com/walkguide/
│ │ ├── config/ # SecurityConfig, WebSocketConfig, OpenApiConfig, FcmConfig, DataSeeder
│ │ ├── enums/ # UserRole, PairingStatus, ActivityLogType, VoiceCommandKey,
│ │ │ # HardwareShortcutKey, NotificationType, SosStatus
│ │ ├── entity/ # User, PairingRelation, ActivityLog, ObstacleLog,
│ │ │ # LocationHistory, GuardianNotification, SosEvent,
│ │ │ # UserSettings, AiConfig, VoiceCommandConfig,
│ │ │ # HardwareShortcut, GeofenceConfig, RefreshToken
│ │ ├── repository/ # JPA repositories for all entities
│ │ ├── dto/
│ │ │ ├── request/ # RegisterRequest, LoginRequest, InviteUserRequest,
│ │ │ │ # LocationUpdateRequest, ObstacleLogRequest,
│ │ │ │ # SendNotificationRequest, SosRequest,
│ │ │ │ # AiConfigUpdateRequest, VoiceCommandUpdateRequest,
│ │ │ │ # HardwareShortcutUpdateRequest, GeofenceConfigRequest,
│ │ │ │ # UserSettingsUpdateRequest, and more
│ │ │ └── response/ # ApiResponse (standard wrapper), AuthDataResponse,
│ │ │ # UserProfileResponse, PairingStatusResponse,
│ │ │ # ActivityLogResponse, ObstacleLogResponse,
│ │ │ # LocationResponse, NotificationResponse,
│ │ │ # SosEventResponse, AgoraTokenResponse,
│ │ │ # AiConfigResponse, VoiceCommandResponse,
│ │ │ # HardwareShortcutResponse, GeofenceResponse
│ │ ├── service/ # AuthService, PairingService, ActivityLogService,
│ │ │ # LocationService, ObstacleLogService,
│ │ │ # NotificationService, SosService,
│ │ │ # AiConfigService, VoiceCommandService,
│ │ │ # HardwareShortcutService, GeofenceService,
│ │ │ # UserSettingsService, FcmService,
│ │ │ # AgoraTokenService, GuardianDashboardService
│ │ ├── controller/ # AuthController, PairingController,
│ │ │ # GuardianController, UserController, CallController
│ │ ├── security/ # JwtUtil, JwtAuthFilter, CustomUserDetailsService
│ │ ├── websocket/ # LocationBroadcaster (STOMP broadcaster)
│ │ └── exception/ # GlobalExceptionHandler, ResourceNotFoundException,
│ │ # UnauthorizedException, PairingException
│ ├── src/main/resources/
│ │ ├── application.properties # DB config (PostgreSQL 202.46.28.160:2002), Flyway, JWT
│ │ ├── firebase/
│ │ │ └── google-services-admin.json # FCM service account key (gitignored!)
│ │ └── db/migration/
│ │ ├── V1__create_users_table.sql ✅ Exists
│ │ ├── V2__seed_users.sql ✅ Exists
│ │ ├── V3__link_guardian_user.sql ✅ Exists
│ │ ├── V4__add_unique_user_id.sql 🔄 Needs creation
│ │ ├── V5__create_pairing_relations.sql 🔄 Needs creation
│ │ ├── V6__create_activity_logs.sql 🔄 Needs creation
│ │ ├── V7__create_obstacle_logs.sql 🔄 Needs creation
│ │ ├── V8__create_location_history.sql 🔄 Needs creation
│ │ ├── V9__create_notifications.sql 🔄 Needs creation
│ │ ├── V10__create_sos_events.sql 🔄 Needs creation
│ │ ├── V11__create_user_settings.sql 🔄 Needs creation
│ │ ├── V12__create_ai_configs.sql 🔄 Needs creation
│ │ ├── V13__create_voice_commands.sql 🔄 Needs creation
│ │ ├── V14__create_hardware_shortcuts.sql 🔄 Needs creation
│ │ ├── V15__create_geofence_configs.sql 🔄 Needs creation
│ │ └── V16__create_refresh_tokens.sql 🔄 Needs creation
│ └── src/test/java/com/walkguide/
│ ├── service/ # AuthServiceTest, PairingServiceTest,
│ │ # NotificationServiceTest, SosServiceTest,
│ │ # LocationServiceTest, AiConfigServiceTest
│ └── controller/ # AuthControllerTest, GuardianControllerTest,
│ # UserControllerTest (MockMvc + Testcontainers)
├── walkguide-mobile/ # [Flutter Frontend — PENDING]
│ ├── lib/
│ │ ├── main.dart # Firebase init, GetIt setup, camera init
│ │ ├── app/
│ │ │ ├── app.dart # MultiBlocProvider + MaterialApp.router
│ │ │ ├── router.dart # GoRouter: /server-connect → /splash → role-based routes
│ │ │ └── injection_container.dart # GetIt singletons & factories
│ │ ├── core/
│ │ │ ├── constants/ # app_constants.dart (dynamic BASE_URL via SharedPreferences),
│ │ │ │ # route_constants.dart, voice_command_keys.dart
│ │ │ ├── errors/ # failures.dart, exceptions.dart
│ │ │ ├── network/ # api_client.dart (Dio), auth_interceptor.dart,
│ │ │ │ # error_interceptor.dart, network_info.dart
│ │ │ ├── storage/ # secure_storage.dart (JWT), local_database.dart (Drift)
│ │ │ ├── services/ # tts_service.dart, stt_service.dart,
│ │ │ │ # voice_command_handler.dart,
│ │ │ │ # hardware_shortcut_listener.dart,
│ │ │ │ # fcm_service.dart, haptic_service.dart,
│ │ │ │ # websocket_service.dart, agora_service.dart
│ │ │ ├── ai/ # model_loader.dart, yolo_detector.dart,
│ │ │ │ # obstacle_analyzer.dart
│ │ │ ├── theme/ # app_theme.dart, app_colors.dart
│ │ │ └── utils/ # permission_manager.dart, location_service.dart
│ │ ├── features/
│ │ │ ├── server_connect/ # ServerConnectScreen — FIRST SCREEN on fresh install
│ │ │ │ # BLoC: TestConnection, SaveAndContinue, ChangeServer
│ │ │ ├── auth/ # SplashScreen, LoginScreen, RegisterScreen
│ │ │ │ # Clean Arch: domain / data / presentation
│ │ │ ├── pairing/ # UserPairingScreen (show unique ID, accept invite)
│ │ │ │ # GuardianPairingScreen (invite by 12-char ID)
│ │ │ ├── walk_guide/ # WalkGuideScreen (full-screen camera + YOLO overlay)
│ │ │ │ # WalkGuideBloc: Start → camera stream → detect → TTS
│ │ │ ├── sos/ # SosScreen (large SOS button)
│ │ │ ├── activity_log/ # ActivityLogScreen (paginated, filterable)
│ │ │ ├── notifications/ # NotificationScreen + TTS read-all feature
│ │ │ ├── navigation_mode/ # NavigationModeScreen (OSM map + OSRM routing)
│ │ │ ├── settings/ # UserSettingsScreen (TTS, pairing, account)
│ │ │ ├── call/ # CallScreen + IncomingCallScreen (Agora VoIP)
│ │ │ ├── manual/ # ManualScreen (all voice commands & shortcuts)
│ │ │ └── guardian_dashboard/ # GuardianDashboardScreen, GuardianMapScreen,
│ │ │ # GuardianActivityLogScreen, GuardianSendNotifScreen,
│ │ │ # GuardianAiConfigScreen, GuardianVoiceCmdScreen,
│ │ │ # GuardianShortcutScreen, GuardianGeofenceScreen
│ │ └── shared/widgets/ # AppBottomNav, VoiceCommandListener,
│ │ # TtsAwareScaffold, LoadingOverlay, ErrorWidget
│ ├── assets/
│ │ ├── models/
│ │ │ ├── yolov8n.tflite # YOLOv8n converted to TFLite (~6MB)
│ │ │ └── labels.txt # 80 COCO object labels
│ │ └── images/
│ └── pubspec.yaml
├── ooad-docs/ # [Design Artifacts]
│ ├── diagrams/ # PlantUML / draw.io exports
│ └── Traceability_Matrix.md
└── README.md
|-- walkguide-backend/
| |-- demo/
| |-- src/main/java/com/walkguide/
| | |-- config/
| | |-- controller/
| | |-- dto/
| | |-- entity/
| | |-- enums/
| | |-- exception/
| | |-- repository/
| | |-- security/
| | |-- service/
| | `-- websocket/
| |-- src/main/resources/
| | |-- application.properties
| | |-- application-dev.yml
| | |-- application-prod.yml
| | `-- db/migration/V1...V17
| |-- src/test/java/com/walkguide/
| |-- k6-tests/
| `-- pom.xml
|
|-- walkguide-mobile/
| `-- walkguide_app/
| |-- lib/
| | |-- app/
| | |-- core/
| | |-- features/
| | `-- shared/
| |-- assets/
| | |-- images/
| | `-- models/
| |-- test/
| |-- integration_test/
| `-- pubspec.yaml
|
|-- ooad-docs/
| |-- 01_Builder_Pattern.puml
| |-- 02_Singleton_Pattern.puml
| |-- 03_Facade_Pattern.puml
| |-- 04_Repository_Proxy_Pattern.puml
| |-- 05_Observer_Pattern.puml
| |-- 06_Strategy_Pattern.puml
| |-- 07_ChainOfResponsibility_Pattern.puml
| `-- diagrams/
|
|-- FULL_FLOW_ARCHITECTURE.md
|-- FINAL_EXAM_GUIDE.md
|-- TODO.md
`-- README.md
```
---
## Feature Flows
## Feature Flows (High-Level)
### Flow 1 - Register And Pairing
### Flow 1 — Register & Pairing
Guardian registers → User registers (gets a 12-char `uniqueUserId`) → Guardian enters User's ID → backend sends FCM invite → User accepts in app → backend seeds 14 default voice commands, 5 hardware shortcuts, and AI config for the pair.
Guardian/User registers -> backend creates account and role data -> User receives pairing identity/code -> Guardian sends invite -> User responds -> backend activates pairing and seeds related configs.
### Flow 2 — WalkGuide Active Detection
User says "Start Walkguide" (or presses Volume Down) → camera starts → frames are throttled to max inference FPS → YOLOv8n detects obstacles → direction and distance are analyzed → TTS announces "Caution! Person ahead center. Very close. Please stop." → haptic feedback triggers → obstacle logged to backend → GPS location broadcast via WebSocket to Guardian's live map every 5 seconds.
### Flow 2 - WalkGuide Detection
### Flow 3 — VoIP Call
User says "Call Guardian" → Agora token fetched from backend → FCM "Incoming Call" sent to Guardian → Guardian accepts in `IncomingCallScreen` → both join same Agora RTC channel → audio call connects.
User starts WalkGuide -> camera/AI pipeline runs on device -> obstacle results are analyzed -> TTS/haptic feedback is emitted -> obstacle/location events can be logged to backend.
### Flow 4 — Guardian Sends Notification → User Hears via TTS
Guardian sends text/voice note → backend saves notification + sends FCM → User receives in foreground via WebSocket → `flutter_local_notifications` shown + TTS announced → User says "Read All My Notifications" → TTS reads each message → marked as read.
### Flow 3 - SOS Alert
### Flow 5 — SOS Alert
User says "Send SOS" (or hardware shortcut) → GPS captured → backend saves SOS event → high-priority FCM sent to Guardian → Guardian map opens to User's location → Guardian taps Acknowledge → FCM sent back to User → TTS "Guardian is on the way."
User triggers SOS -> backend creates SOS event -> Guardian receives/loads SOS event -> Guardian taps acknowledge or resolve -> SOS status no longer piles up as unhandled.
### Flow 6 — Geofence
Guardian sets center + radius on map → User's location updates checked against Haversine distance → on exit: FCM to Guardian + `GEOFENCE_EXIT` activity log.
### Flow 4 - Guardian Notification
---
Guardian opens send notification -> chooses text or voice note mode -> sends payload -> backend stores notification -> User notification screen can display/read/play supported message data.
## Quick Start (WIP)
### Flow 5 - Location And Map
```bash
# 1. Clone the repository
git clone https://github.com/YourGroup/walkguide-final-exam.git
User sends location updates -> backend stores location history -> Guardian map/dashboard reads the latest location and history -> WebSocket support exists for real-time updates.
# 2. Run Backend (Spring Boot)
cd walkguide-backend
./mvnw spring-boot:run
# Flyway auto-migrates V1V16 on first start
# Swagger UI: http://202.46.28.160:8080/swagger-ui.html
# Health check: GET http://202.46.28.160:8080/api/v1/auth/ping
### Flow 6 - Call
# 3. Run Mobile (Flutter) - Connect to local or university backend
cd ../walkguide-mobile
Caller requests token/channel -> backend returns call token payload -> caller notifies target -> target receives incoming call flow -> call can be ended through shared endpoint.
## Quick Start
### Backend
```powershell
cd "D:\CodeSpace\Final Project Gabungan - Broken Test\walkguide-backend\demo"
.\mvnw.cmd spring-boot:run -Dspring-boot.run.profiles=dev
```
Health check:
```text
http://localhost:8080/api/v1/auth/ping
```
Swagger UI:
```text
http://localhost:8080/swagger-ui.html
```
The local/dev profile has fallback DB and JWT values. If you want to override them:
```powershell
$env:DB_URL="jdbc:postgresql://202.46.28.160:2002/uas_5803024001"
$env:DB_USERNAME="5803024001"
$env:DB_PASSWORD="pw5803024001"
$env:JWT_SECRET="your-base64-secret"
```
### Flutter
```powershell
cd "D:\CodeSpace\Final Project Gabungan - Broken Test\walkguide-mobile\walkguide_app"
flutter pub get
flutter run
# On first launch: enter server URL in ServerConnectScreen
# Example: http://202.46.28.160:8080
```
---
For Chrome/web debug:
```powershell
flutter run -d chrome -t lib/main.dart
```
For Android APK:
```powershell
flutter build apk --release
```
On a physical phone, the server URL must be reachable by the phone. Use the university server URL or your laptop LAN IP, not `localhost`.
## Results
> ⏳ **Work In Progress:** Results are populated as benchmarking phases are completed.
Final benchmark values should be filled from real test runs before submission.
| Metric | Baseline | Final Optimized | Status |
|---|---|---|---|
| Cold Start Time | *Pending* | *Pending* | ⏳ |
| Memory Leak (10 Navs) | *Pending* | *Pending* | ⏳ |
| API Error Rate | *Pending* | *Pending* | ⏳ |
| YOLO Inference Latency (ms) | *Pending* | *Pending* | ⏳ |
| API p95 Latency | *Pending* | *Pending* | ⏳ |
| JaCoCo Coverage | *Pending* | *Pending* | ⏳ |
---
| Metric | Evidence Location / Tool | Current README Status |
|---|---|---|
| Backend tests | Maven/JUnit output | To be generated |
| Backend coverage | JaCoCo report | To be generated |
| Backend load | k6 results | Assets present, final run needed |
| Flutter tests | `flutter test` / integration tests | To be generated |
| Flutter performance | Physical Android profile evidence | To be generated |
| APK size | `flutter build apk --analyze-size` | To be generated |
## Weekly Progress
| Week | Target | Status |
| Week | Target | Current Status |
|---|---|---|
| 1 | Topic proposal, Use Case definitions, Repo setup | ✅ Done |
| 23 | OOAD diagrams, OpenAPI YAML drafted | 🔄 In Progress |
| 4 | Spring Boot: Auth, Pairing, Entity, Migration V4V16 | 🔄 In Progress |
| 5 | Spring Boot: Location, SOS, Notification, WS, FCM, Agora | ⏳ Pending |
| 6 | Spring Boot: Unit + Integration tests, JaCoCo ≥ 70% | ⏳ Pending |
| 7 | Flutter: ServerConnect, Auth, WalkGuide + YOLO pipeline | ⏳ Pending |
| 8 | Flutter: Guardian dashboard, Call, SOS, Notifications | ⏳ Pending |
| 9 | Feature freeze, integration testing, benchmark on device | ⏳ Pending |
| 10 | Final benchmarks, Report writing, Demo Video | ⏳ Pending |
---
| 1 | Topic proposal, use case definitions, repo setup | Done |
| 2-3 | OOAD diagrams and traceability | Done, docs present in `ooad-docs/` |
| 4 | Spring Boot auth, pairing, entities, migrations | Implemented |
| 5 | Location, SOS, notification, WebSocket, FCM, call support | Implemented with demo/service integrations |
| 6 | Backend unit/integration testing and coverage setup | Implemented, final run evidence needed |
| 7 | Flutter server connect, auth, WalkGuide/YOLO support | Implemented |
| 8 | Guardian dashboard, SOS, notifications, voice notes, settings | Implemented |
| 9 | Integration testing and benchmark evidence | Needs final evidence run |
| 10 | Report, demo video, final submission polish | In progress |
## Core Objectives
**O₁ (Performance):** The dual-platform architecture must maintain a 60fps UI frame rate while handling concurrent backend requests without exceeding 500ms latency. YOLO inference runs on a separate isolate to avoid blocking the UI thread.
O1 Performance: Keep obstacle detection local/on-device where possible and use backend for persistence, pairing, configuration, and real-time coordination.
**O₂ (Accessibility):** The User mode must require zero visual interaction post-login, relying entirely on haptics, voice commands, and physical button mapping. All screen transitions are announced via TTS.
O2 Accessibility: Support voice/TTS, haptic feedback, large touch targets, and hardware shortcut flows for visually impaired users.
**O₃ (Traceability):** Every major feature implementation must be directly traceable back to the pre-development OOAD artifacts and GoF design patterns.
O3 Traceability: Keep implementation aligned with `FULL_FLOW_ARCHITECTURE.md`, `FINAL_EXAM_GUIDE.md`, and the PUML diagrams in `ooad-docs/`.
**O₄ (Configurability):** All AI sensitivity settings, voice command phrases, and hardware shortcuts are remotely configurable by the Guardian without requiring an app update.
---
O4 Configurability: Let Guardian configure AI sensitivity, voice commands, hardware shortcuts, geofence, and User settings through dashboard flows.
## License
Distributed under the [MIT License](LICENSE).
---
*Final Exam: Integrated Mobile Application Project* <br>
*Flutter × Spring Boot × Object-Oriented Analysis and Design*
Final Exam: Integrated Mobile Application Project
Flutter x Spring Boot x Object-Oriented Analysis and Design