Final-08-Evan-Jap-Bambang-WalkGuide-AI/README.md

# WalkGuide: AI-Powered Navigation

Integrated Mobile Application Project
Flutter Mobile Frontend x Spring Boot Backend x OOAD

WalkGuide is an accessibility-focused mobile system for visually impaired users and their guardians. The User app provides camera-based obstacle awareness, voice/TTS interaction, SOS, notifications, navigation, and call flows. The Guardian app provides monitoring, live location, SOS handling, remote configuration, notification sending, voice notes, and pairing management.

## Group Members

| Name | NIM | Responsibility |
|---|---:|---|
| Bambang Herlambang | 5803024019 | Mobile feature support, documentation, testing |
| Jap Robertus | 5803024004 | Mobile feature support, documentation, testing |
| Evan William | 5803024001 | Backend API, Flutter integration, architecture alignment |

## Project Status

This repository is no longer a backend-only skeleton. The current codebase contains:

- Spring Boot backend with JWT/RBAC, Flyway migrations V1-V17, PostgreSQL, OpenAPI, WebSocket, FCM service hooks, Agora token/call notification flow, SOS acknowledge/resolve flow, pairing code flow, service/controller tests, Testcontainers setup, and k6 assets.
- Flutter app with server connection screen, auth, role-based routing, User screens, Guardian screens, feature page UI shell, friendly error handling, offline queue/cache layer, voice note UI support, SOS handling UI, WalkGuide/YOLO support files, and Android/mobile-first dependencies.
- OOAD documentation in `ooad-docs/`, including the 7 GoF design pattern PUML diagrams and traceability documentation.

Primary demo target: Android APK connected to the Spring Boot backend.
Chrome/web can be used for UI/debug flows, but camera, native AI, SQLite FFI, and mobile permissions are Android-first.

## Overview

Core objective: build an accessible navigation assistant that can help visually impaired users move more safely while allowing a Guardian to monitor, configure, and respond to events in real time.

Important flows implemented or represented in the codebase:

- Register/Login with Guardian and User roles.
- Pairing by generated pairing code / user identity flow.
- Guardian dashboard and tools screens.
- User SOS trigger, Guardian acknowledge and resolve.
- Guardian text and voice-note style notification sending.
- Notification read/read-all handling.
- Location update and Guardian live map support.
- AI configuration, voice command configuration, hardware shortcut configuration.
- WalkGuide obstacle detection pipeline integration points.
- Call token and call notification endpoints for Agora-style VoIP flow.

## System Architecture

The project follows a feature-first architecture across backend, mobile, and OOAD documents.

### Backend

Backend follows layered Spring Boot architecture:

```text
Controller -> Service -> Repository -> Entity -> PostgreSQL
```

Main backend concerns:

- `controller/`: REST API endpoints.
- `service/`: application/business logic.
- `repository/`: Spring Data JPA persistence contracts.
- `entity/`: database-mapped entities.
- `dto/request` and `dto/response`: API input/output contracts.
- `security/`: JWT utility, JWT filter, custom user details service.
- `config/`: Security, WebSocket, OpenAPI, Firebase/FCM, seeding.
- `websocket/`: STOMP broadcasting helper.
- `db/migration/`: Flyway schema migrations.

### Flutter

Flutter uses a feature-first layout. Several critical features now have domain/data/application/presentation structure, while compatibility wrapper screens remain for the existing app routes.

Main Flutter concerns:

- `app/`: app shell, router, dependency injection.
- `core/`: API service/client, services, storage/cache, AI helpers, errors.
- `features/`: auth, server connect, pairing, SOS, notifications, WalkGuide, activity log, navigation, settings, call, guardian dashboard/tools, manual, benchmark.
- `shared/widgets/`: common UI shell and reusable feature page components.

### OOAD

The `ooad-docs/` folder contains traceability and diagrams, including:

- `01_Builder_Pattern.puml`
- `02_Singleton_Pattern.puml`
- `03_Facade_Pattern.puml`
- `04_Repository_Proxy_Pattern.puml`
- `05_Observer_Pattern.puml`
- `06_Strategy_Pattern.puml`
- `07_ChainOfResponsibility_Pattern.puml`
- Use case, sequence, state, ERD, class, and component diagrams under `ooad-docs/diagrams/`.

## Tech Stack

### Backend

| Tool / Library | Current Codebase | Purpose |
|---|---|---|
| Java | 21 | Backend language |
| Spring Boot | 3.2.5 | Main backend framework |
| Spring Security | Spring Boot managed | JWT auth and RBAC |
| Spring Data JPA | Spring Boot managed | ORM and repositories |
| Spring WebSocket | Spring Boot managed | STOMP realtime channels |
| PostgreSQL Driver | Runtime dependency | University PostgreSQL database |
| Flyway | Core + PostgreSQL module | Database migrations |
| JJWT | 0.11.5 | JWT access token handling |
| Springdoc OpenAPI | 2.3.0 | Swagger/OpenAPI docs |
| Lombok | 1.18.36 | Builders and boilerplate reduction |
| JUnit 5 / Mockito / MockMvc | Test dependencies | Unit and controller testing |
| Testcontainers | 1.19.7 | PostgreSQL-backed integration tests |
| JaCoCo | 0.8.11 | Coverage report |

### Flutter

| Package | Current Codebase | Purpose |
|---|---|---|
| flutter_bloc | 8.1.6 | Cubit/BLoC state management |
| go_router | 14.2.7 | App routing |
| dio | 5.4.3+1 | REST client |
| flutter_secure_storage | 9.2.2 | Secure token storage on mobile |
| shared_preferences | 2.3.2 | Server URL and web cache fallback |
| drift / sqlite3 | drift 2.18.0, sqlite3 2.4.7 | Local persistence support |
| sqlite3_flutter_libs | 0.5.24 | Android/iOS SQLite native libs |
| camera | 0.11.0+2 | Camera feed |
| tflite_flutter | 0.12.1 | On-device model inference |
| flutter_tts | 4.0.2 | Text-to-speech |
| speech_to_text | 7.0.0 | Voice recognition support |
| firebase_core / firebase_messaging | 3.3.0 / 15.1.0 | FCM integration |
| flutter_local_notifications | 17.2.1+2 | Foreground/local notification UI |
| flutter_map / latlong2 | 7.0.2 / 0.9.1 | OpenStreetMap UI |
| geolocator | 12.0.0 | GPS/location |
| agora_rtc_engine | 6.3.2 | VoIP engine integration |
| just_audio / record | 0.9.40 / 5.1.2 | Voice note playback/recording |
| get_it | 8.0.2 | Dependency injection |
| dartz | 0.10.1 | Either-style error handling |
| connectivity_plus | 6.0.3 | Offline/online detection |

## Runtime Configuration

### Backend

Local/dev config imports an optional gitignored file at `walkguide-backend/demo/secrets.properties`.
Copy `walkguide-backend/demo/secrets.properties.example` to `secrets.properties` and fill it locally.
Tracked config files do not contain DB passwords, JWT secrets, Agora certificates, or Firebase keys.

Both dev and production expect these values from environment variables or `secrets.properties`:

```text
DB_URL
DB_USERNAME
DB_PASSWORD
JWT_SECRET
AGORA_APP_ID
AGORA_APP_CERTIFICATE
```

### Flutter

The Flutter app uses a dynamic server URL. On first launch, use the Server Connect screen and enter a backend URL such as:

```text
http://202.46.28.170:8080
```

For testing on a physical phone against a backend running on a laptop, do not use `localhost`. Use the laptop LAN IP:

```text
http://192.168.x.x:8080
```

## Implementations

### User Mode

- WalkGuide camera screen and AI pipeline integration points.
- TTS feedback and friendly error handling.
- SOS screen with emergency action flow.
- Notifications screen with read/read-all actions.
- Activity log, navigation, settings, pairing, and manual screens.
- Voice command and shortcut configuration retrieval paths.
- Offline queue/cache support for core app data.

### Guardian Mode

- Guardian dashboard and tools.
- Guardian live map screen.
- Guardian activity log screen.
- Send notification screen with text and voice note modes.
- AI config screen.
- Voice command and shortcut management screens.
- Geofence/settings screens.
- SOS acknowledge and resolve support.

### Backend API

- Auth, pairing, user, guardian, and shared call controllers.
- Service layer for pairing, location, activity, obstacles, notifications, SOS, AI config, voice commands, hardware shortcuts, geofence, user settings, call notifications, and dashboard aggregation.
- WebSocket broadcaster for location/SOS/notification-style real-time updates.
- Flyway-managed PostgreSQL schema.

## Database Schema

Database: PostgreSQL on the university server.
Schema management: Flyway.
Hibernate mode: `validate`.

Current migrations in `walkguide-backend/demo/src/main/resources/db/migration/`:

| Migration | File | Status |
|---|---|---|
| V1 | `V1__create_users_table.sql` | Present |
| V2 | `V2__seed_users.sql` | Present |
| V3 | `V3__link_guardian_user.sql` | Present |
| V4 | `V4__alter_users_add_columns.sql` | Present |
| V5 | `V5__create_pairing_relations.sql` | Present |
| V6 | `V6__create_activity_logs.sql` | Present |
| V7 | `V7__create_obstacle_logs.sql` | Present |
| V8 | `V8__create_location_history.sql` | Present |
| V9 | `V9__create_guardian_notifications.sql` | Present |
| V10 | `V10__create_sos_events.sql` | Present |
| V11 | `V11__create_user_settings.sql` | Present |
| V12 | `V12__create_ai_configs.sql` | Present |
| V13 | `V13__create_voice_command_configs.sql` | Present |
| V14 | `V14__create_hardware_shortcuts.sql` | Present |
| V15 | `V15__create_geofence_configs.sql` | Present |
| V16 | `V16__create_refresh_tokens.sql` | Present |
| V17 | `V17__add_expiring_pairing_codes.sql` | Present |

## API Endpoints

### Auth - `/api/v1/auth`

| Method | Path | Description |
|---|---|---|
| GET | `/ping` | Server connection check |
| POST | `/register` | Register Guardian or User |
| POST | `/login` | Login |
| POST | `/refresh` | Refresh access token |
| POST | `/logout` | Logout |
| PUT | `/fcm-token` | Update FCM token |

### Pairing - `/api/v1/shared/pairing`

| Method | Path | Description |
|---|---|---|
| GET | `/code` | Get current pairing code |
| POST | `/code/regenerate` | Regenerate pairing code |
| POST | `/invite` | Guardian invites User |
| POST | `/respond` | User accepts/rejects invite |
| DELETE | `/unpair` | Remove pairing |
| GET | `/status` | Get current pairing status |

### Guardian - `/api/v1/guardian`

| Method | Path | Description |
|---|---|---|
| GET | `/dashboard` | Guardian home data |
| GET | `/user-location` | Last known User location |
| GET | `/location-history` | Paginated location history |
| GET | `/activity-logs` | Paginated User activity logs |
| GET | `/obstacle-logs` | Paginated obstacle logs |
| POST | `/notifications/send` | Send text or voice note notification |
| GET | `/sos-events` | Paginated SOS events |
| PUT | `/sos/{id}/acknowledge` | Mark SOS as acknowledged |
| PUT | `/sos/{id}/resolve` | Mark SOS as resolved/handled |
| GET/PUT | `/ai-config` | Get/update AI config |
| GET/PUT | `/voice-commands` | Get/update voice command config |
| GET/PUT | `/shortcuts` | Get/update shortcut config |
| GET/PUT | `/geofence` | Get/update geofence config |
| GET/PUT | `/user-settings` | Get/update paired User settings |

### User - `/api/v1/user`

| Method | Path | Description |
|---|---|---|
| GET | `/profile` | Current User profile |
| GET/PUT | `/settings` | Get/update User settings |
| GET | `/voice-commands` | Get voice command config |
| GET/PUT | `/shortcuts` | Get/update shortcut config |
| GET | `/ai-config` | Get AI config |
| POST | `/location` | Send location update |
| POST | `/obstacle` | Log obstacle |
| POST | `/sos` | Trigger SOS |
| GET | `/sos-events` | Get own SOS events |
| GET | `/activity-logs` | Get activity logs |
| GET | `/notifications` | Get notifications |
| GET | `/notifications/unread-count` | Get unread count |
| PUT | `/notifications/mark-all-read` | Mark all notifications read |
| PUT | `/notifications/{id}/read` | Mark one notification read |
| POST | `/walkguide/start` | Log WalkGuide start |
| POST | `/walkguide/stop` | Log WalkGuide stop |

### Shared Call - `/api/v1/shared/call`

| Method | Path | Description |
|---|---|---|
| POST | `/token` | Generate call token/channel payload |
| POST | `/notify` | Notify other party of incoming call |
| POST | `/end` | Notify/end call session |

## Design Patterns

The project documents and maps 7 GoF-related patterns in `ooad-docs/`.

| # | Category | Pattern | Main Location |
|---|---|---|---|
| 1 | Creational | Builder | Backend entities/DTO construction, Lombok builders |
| 2 | Creational | Singleton | Flutter services registered through GetIt |
| 3 | Structural | Facade | Guardian dashboard aggregation and voice command/service coordination |
| 4 | Structural | Repository / Proxy | Spring Data repositories and Flutter repository-style data access |
| 5 | Behavioral | Observer | BLoC/Cubit state listeners and WebSocket callbacks |
| 6 | Behavioral | Strategy | Obstacle analysis / alert behavior mapping documented in OOAD |
| 7 | Behavioral | Chain of Responsibility | Spring Security filter chain and Dio interceptors |

See:

- `ooad-docs/DESIGN_PATTERNS.md`
- `ooad-docs/TRACEABILITY_AUDIT.md`
- `ooad-docs/01_Builder_Pattern.puml` through `ooad-docs/07_ChainOfResponsibility_Pattern.puml`

## Metrics And Evidence

The repository includes testing and benchmark support, but final scoring evidence should be generated on the target machine/device before submission.

| Area | Current Support |
|---|---|
| Backend unit tests | JUnit/Mockito tests under `src/test/java` |
| Backend controller/integration tests | MockMvc and Testcontainers setup present |
| Coverage | JaCoCo configured in Maven |
| Load testing | k6 assets/results folder present under backend project |
| Flutter tests | Unit/widget/integration test files present |
| Flutter performance | Benchmark evidence template in `ooad-docs/BENCHMARK_EVIDENCE_TEMPLATE.md` |

Recommended final evidence:

- `mvn test` or `mvn verify` output.
- JaCoCo HTML report.
- Testcontainers run on a Docker-enabled machine.
- k6 result at 50 or more concurrent virtual users.
- Flutter physical Android profile evidence: cold start, memory, jank/frame timing, CPU during AI, API latency, APK size.

## Repository Structure

```text
/
|-- walkguide-backend/
|   |-- demo/
|       |-- src/main/java/com/walkguide/
|       |   |-- config/
|       |   |-- controller/
|       |   |-- dto/
|       |   |-- entity/
|       |   |-- enums/
|       |   |-- exception/
|       |   |-- repository/
|       |   |-- security/
|       |   |-- service/
|       |   `-- websocket/
|       |-- src/main/resources/
|       |   |-- application.properties
|       |   |-- application-dev.yml
|       |   |-- application-prod.yml
|       |   `-- db/migration/V1...V17
|       |-- src/test/java/com/walkguide/
|       |-- k6-tests/
|       `-- pom.xml
|
|-- walkguide-mobile/
|   `-- walkguide_app/
|       |-- lib/
|       |   |-- app/
|       |   |-- core/
|       |   |-- features/
|       |   `-- shared/
|       |-- assets/
|       |   |-- images/
|       |   `-- models/
|       |-- test/
|       |-- integration_test/
|       `-- pubspec.yaml
|
|-- ooad-docs/
|   |-- 01_Builder_Pattern.puml
|   |-- 02_Singleton_Pattern.puml
|   |-- 03_Facade_Pattern.puml
|   |-- 04_Repository_Proxy_Pattern.puml
|   |-- 05_Observer_Pattern.puml
|   |-- 06_Strategy_Pattern.puml
|   |-- 07_ChainOfResponsibility_Pattern.puml
|   `-- diagrams/
|
|-- FULL_FLOW_ARCHITECTURE.md
|-- FINAL_EXAM_GUIDE.md
|-- TODO.md
`-- README.md
```

## Feature Flows

### Flow 1 - Register And Pairing

Guardian/User registers -> backend creates account and role data -> User receives pairing identity/code -> Guardian sends invite -> User responds -> backend activates pairing and seeds related configs.

### Flow 2 - WalkGuide Detection

User starts WalkGuide -> camera/AI pipeline runs on device -> obstacle results are analyzed -> TTS/haptic feedback is emitted -> obstacle/location events can be logged to backend.

### Flow 3 - SOS Alert

User triggers SOS -> backend creates SOS event -> Guardian receives/loads SOS event -> Guardian taps acknowledge or resolve -> SOS status no longer piles up as unhandled.

### Flow 4 - Guardian Notification

Guardian opens send notification -> chooses text or voice note mode -> sends payload -> backend stores notification -> User notification screen can display/read/play supported message data.

### Flow 5 - Location And Map

User sends location updates -> backend stores location history -> Guardian map/dashboard reads the latest location and history -> WebSocket support exists for real-time updates.

### Flow 6 - Call

Caller requests token/channel -> backend returns call token payload -> caller notifies target -> target receives incoming call flow -> call can be ended through shared endpoint.

## Quick Start

### Backend

```powershell
cd "D:\CodeSpace\Final Project Gabungan - Broken Test\walkguide-backend\demo"
.\mvnw.cmd spring-boot:run -Dspring-boot.run.profiles=dev
```

Health check:

```text
http://localhost:8080/api/v1/auth/ping
```

Swagger UI:

```text
http://localhost:8080/swagger-ui.html
```

Local/dev configuration reads database and secret values from environment variables or from the gitignored `secrets.properties` file:

```powershell
$env:DB_URL="jdbc:postgresql://<host>:<port>/<database>"
$env:DB_USERNAME="<database_username>"
$env:DB_PASSWORD="<database_password>"
$env:JWT_SECRET="your-base64-secret"
```

### Flutter

```powershell
cd "D:\CodeSpace\Final Project Gabungan - Broken Test\walkguide-mobile\walkguide_app"
flutter pub get
flutter run
```

For Chrome/web debug:

```powershell
flutter run -d chrome -t lib/main.dart
```

For Android APK:

```powershell
flutter build apk --release
```

On a physical phone, the server URL must be reachable by the phone. Use the university server URL or your laptop LAN IP, not `localhost`.

## Results

Final benchmark values should be filled from real test runs before submission.

| Metric | Evidence Location / Tool | Current README Status |
|---|---|---|
| Backend tests | Maven/JUnit output | To be generated |
| Backend coverage | JaCoCo report | To be generated |
| Backend load | k6 results | Assets present, final run needed |
| Flutter tests | `flutter test` / integration tests | To be generated |
| Flutter performance | Physical Android profile evidence | To be generated |
| APK size | `flutter build apk --analyze-size` | To be generated |

## Weekly Progress

| Week | Target | Current Status |
|---|---|---|
| 1 | Topic proposal, use case definitions, repo setup | Done |
| 2-3 | OOAD diagrams and traceability | Done, docs present in `ooad-docs/` |
| 4 | Spring Boot auth, pairing, entities, migrations | Implemented |
| 5 | Location, SOS, notification, WebSocket, FCM, call support | Implemented with demo/service integrations |
| 6 | Backend unit/integration testing and coverage setup | Implemented, final run evidence needed |
| 7 | Flutter server connect, auth, WalkGuide/YOLO support | Implemented |
| 8 | Guardian dashboard, SOS, notifications, voice notes, settings | Implemented |
| 9 | Integration testing and benchmark evidence | Needs final evidence run |
| 10 | Report, demo video, final submission polish | In progress |

## Core Objectives

O1 Performance: Keep obstacle detection local/on-device where possible and use backend for persistence, pairing, configuration, and real-time coordination.

O2 Accessibility: Support voice/TTS, haptic feedback, large touch targets, and hardware shortcut flows for visually impaired users.

O3 Traceability: Keep implementation aligned with `FULL_FLOW_ARCHITECTURE.md`, `FINAL_EXAM_GUIDE.md`, and the PUML diagrams in `ooad-docs/`.

O4 Configurability: Let Guardian configure AI sensitivity, voice commands, hardware shortcuts, geofence, and User settings through dashboard flows.

## License

Distributed under the [MIT License](LICENSE).

Final Exam: Integrated Mobile Application Project
Flutter x Spring Boot x Object-Oriented Analysis and Design