WalkGuide: AI-Powered Navigation

Integrated Mobile Application Project Flutter Mobile Frontend x Spring Boot Backend x OOAD

WalkGuide is an accessibility-focused mobile system for visually impaired users and their guardians. The User app provides camera-based obstacle awareness, voice/TTS interaction, SOS, notifications, navigation, and call flows. The Guardian app provides monitoring, live location, SOS handling, remote configuration, notification sending, voice notes, and pairing management.

Group Members

Name	NIM	Responsibility
Bambang Herlambang	5803024019	Mobile feature support, documentation, testing
Jap Robertus	5803024004	Mobile feature support, documentation, testing
Evan William	5803024001	Backend API, Flutter integration, architecture alignment

Project Status

This repository is no longer a backend-only skeleton. The current codebase contains:

• Spring Boot backend with JWT/RBAC, Flyway migrations V1-V17, PostgreSQL, OpenAPI, WebSocket, FCM service hooks, Agora token/call notification flow, SOS acknowledge/resolve flow, pairing code flow, service/controller tests, Testcontainers setup, and k6 assets.
• Flutter app with server connection screen, auth, role-based routing, User screens, Guardian screens, feature page UI shell, friendly error handling, offline queue/cache layer, voice note UI support, SOS handling UI, WalkGuide/YOLO support files, and Android/mobile-first dependencies.
• OOAD documentation in ooad-docs/, including the 7 GoF design pattern PUML diagrams and traceability documentation.

Primary demo target: Android APK connected to the Spring Boot backend. Chrome/web can be used for UI/debug flows, but camera, native AI, SQLite FFI, and mobile permissions are Android-first.

Overview

Core objective: build an accessible navigation assistant that can help visually impaired users move more safely while allowing a Guardian to monitor, configure, and respond to events in real time.

Important flows implemented or represented in the codebase:

• Register/Login with Guardian and User roles.
• Pairing by generated pairing code / user identity flow.
• Guardian dashboard and tools screens.
• User SOS trigger, Guardian acknowledge and resolve.
• Guardian text and voice-note style notification sending.
• Notification read/read-all handling.
• Location update and Guardian live map support.
• AI configuration, voice command configuration, hardware shortcut configuration.
• WalkGuide obstacle detection pipeline integration points.
• Call token and call notification endpoints for Agora-style VoIP flow.

System Architecture

The project follows a feature-first architecture across backend, mobile, and OOAD documents.

Backend

Backend follows layered Spring Boot architecture:

Controller -> Service -> Repository -> Entity -> PostgreSQL

Main backend concerns:

• controller/: REST API endpoints.
• service/: application/business logic.
• repository/: Spring Data JPA persistence contracts.
• entity/: database-mapped entities.
• dto/request and dto/response: API input/output contracts.
• security/: JWT utility, JWT filter, custom user details service.
• config/: Security, WebSocket, OpenAPI, Firebase/FCM, seeding.
• websocket/: STOMP broadcasting helper.
• db/migration/: Flyway schema migrations.

Flutter

Flutter uses a feature-first layout. Several critical features now have domain/data/application/presentation structure, while compatibility wrapper screens remain for the existing app routes.

Main Flutter concerns:

• app/: app shell, router, dependency injection.
• core/: API service/client, services, storage/cache, AI helpers, errors.
• features/: auth, server connect, pairing, SOS, notifications, WalkGuide, activity log, navigation, settings, call, guardian dashboard/tools, manual, benchmark.
• shared/widgets/: common UI shell and reusable feature page components.

OOAD

The ooad-docs/ folder contains traceability and diagrams, including:

• 01_Builder_Pattern.puml
• 02_Singleton_Pattern.puml
• 03_Facade_Pattern.puml
• 04_Repository_Proxy_Pattern.puml
• 05_Observer_Pattern.puml
• 06_Strategy_Pattern.puml
• 07_ChainOfResponsibility_Pattern.puml
• Use case, sequence, state, ERD, class, and component diagrams under ooad-docs/diagrams/.

Tech Stack

Backend

Tool / Library	Current Codebase	Purpose
Java	21	Backend language
Spring Boot	3.2.5	Main backend framework
Spring Security	Spring Boot managed	JWT auth and RBAC
Spring Data JPA	Spring Boot managed	ORM and repositories
Spring WebSocket	Spring Boot managed	STOMP realtime channels
PostgreSQL Driver	Runtime dependency	University PostgreSQL database
Flyway	Core + PostgreSQL module	Database migrations
JJWT	0.11.5	JWT access token handling
Springdoc OpenAPI	2.3.0	Swagger/OpenAPI docs
Lombok	1.18.36	Builders and boilerplate reduction
JUnit 5 / Mockito / MockMvc	Test dependencies	Unit and controller testing
Testcontainers	1.19.7	PostgreSQL-backed integration tests
JaCoCo	0.8.11	Coverage report

Flutter

Package	Current Codebase	Purpose
flutter_bloc	8.1.6	Cubit/BLoC state management
go_router	14.2.7	App routing
dio	5.4.3+1	REST client
flutter_secure_storage	9.2.2	Secure token storage on mobile
shared_preferences	2.3.2	Server URL and web cache fallback
drift / sqlite3	drift 2.18.0, sqlite3 2.4.7	Local persistence support
sqlite3_flutter_libs	0.5.24	Android/iOS SQLite native libs
camera	0.11.0+2	Camera feed
tflite_flutter	0.12.1	On-device model inference
flutter_tts	4.0.2	Text-to-speech
speech_to_text	7.0.0	Voice recognition support
firebase_core / firebase_messaging	3.3.0 / 15.1.0	FCM integration
flutter_local_notifications	17.2.1+2	Foreground/local notification UI
flutter_map / latlong2	7.0.2 / 0.9.1	OpenStreetMap UI
geolocator	12.0.0	GPS/location
agora_rtc_engine	6.3.2	VoIP engine integration
just_audio / record	0.9.40 / 5.1.2	Voice note playback/recording
get_it	8.0.2	Dependency injection
dartz	0.10.1	Either-style error handling
connectivity_plus	6.0.3	Offline/online detection

Runtime Configuration

Backend

Local/dev config imports an optional gitignored file at walkguide-backend/demo/secrets.properties. Copy walkguide-backend/demo/secrets.properties.example to secrets.properties and fill it locally. Tracked config files do not contain DB passwords, JWT secrets, Agora certificates, or Firebase keys.

Both dev and production expect these values from environment variables or secrets.properties:

DB_URL
DB_USERNAME
DB_PASSWORD
JWT_SECRET
AGORA_APP_ID
AGORA_APP_CERTIFICATE

Flutter

The Flutter app uses a dynamic server URL. On first launch, use the Server Connect screen and enter a backend URL such as:

http://202.46.28.170:8080

For testing on a physical phone against a backend running on a laptop, do not use localhost. Use the laptop LAN IP:

http://192.168.x.x:8080

Implementations

User Mode

• WalkGuide camera screen and AI pipeline integration points.
• TTS feedback and friendly error handling.
• SOS screen with emergency action flow.
• Notifications screen with read/read-all actions.
• Activity log, navigation, settings, pairing, and manual screens.
• Voice command and shortcut configuration retrieval paths.
• Offline queue/cache support for core app data.

Guardian Mode

• Guardian dashboard and tools.
• Guardian live map screen.
• Guardian activity log screen.
• Send notification screen with text and voice note modes.
• AI config screen.
• Voice command and shortcut management screens.
• Geofence/settings screens.
• SOS acknowledge and resolve support.

Backend API

• Auth, pairing, user, guardian, and shared call controllers.
• Service layer for pairing, location, activity, obstacles, notifications, SOS, AI config, voice commands, hardware shortcuts, geofence, user settings, call notifications, and dashboard aggregation.
• WebSocket broadcaster for location/SOS/notification-style real-time updates.
• Flyway-managed PostgreSQL schema.

Database Schema

Database: PostgreSQL on the university server. Schema management: Flyway. Hibernate mode: validate.

Current migrations in walkguide-backend/demo/src/main/resources/db/migration/:

Migration	File	Status
V1	V1__create_users_table.sql	Present
V2	V2__seed_users.sql	Present
V3	V3__link_guardian_user.sql	Present
V4	V4__alter_users_add_columns.sql	Present
V5	V5__create_pairing_relations.sql	Present
V6	V6__create_activity_logs.sql	Present
V7	V7__create_obstacle_logs.sql	Present
V8	V8__create_location_history.sql	Present
V9	V9__create_guardian_notifications.sql	Present
V10	V10__create_sos_events.sql	Present
V11	V11__create_user_settings.sql	Present
V12	V12__create_ai_configs.sql	Present
V13	V13__create_voice_command_configs.sql	Present
V14	V14__create_hardware_shortcuts.sql	Present
V15	V15__create_geofence_configs.sql	Present
V16	V16__create_refresh_tokens.sql	Present
V17	V17__add_expiring_pairing_codes.sql	Present

API Endpoints

Auth - /api/v1/auth

Method	Path	Description
GET	/ping	Server connection check
POST	/register	Register Guardian or User
POST	/login	Login
POST	/refresh	Refresh access token
POST	/logout	Logout
PUT	/fcm-token	Update FCM token

Pairing - /api/v1/shared/pairing

Method	Path	Description
GET	/code	Get current pairing code
POST	/code/regenerate	Regenerate pairing code
POST	/invite	Guardian invites User
POST	/respond	User accepts/rejects invite
DELETE	/unpair	Remove pairing
GET	/status	Get current pairing status

Guardian - /api/v1/guardian

Method	Path	Description
GET	/dashboard	Guardian home data
GET	/user-location	Last known User location
GET	/location-history	Paginated location history
GET	/activity-logs	Paginated User activity logs
GET	/obstacle-logs	Paginated obstacle logs
POST	/notifications/send	Send text or voice note notification
GET	/sos-events	Paginated SOS events
PUT	/sos/{id}/acknowledge	Mark SOS as acknowledged
PUT	/sos/{id}/resolve	Mark SOS as resolved/handled
GET/PUT	/ai-config	Get/update AI config
GET/PUT	/voice-commands	Get/update voice command config
GET/PUT	/shortcuts	Get/update shortcut config
GET/PUT	/geofence	Get/update geofence config
GET/PUT	/user-settings	Get/update paired User settings

User - /api/v1/user

Method	Path	Description
GET	/profile	Current User profile
GET/PUT	/settings	Get/update User settings
GET	/voice-commands	Get voice command config
GET/PUT	/shortcuts	Get/update shortcut config
GET	/ai-config	Get AI config
POST	/location	Send location update
POST	/obstacle	Log obstacle
POST	/sos	Trigger SOS
GET	/sos-events	Get own SOS events
GET	/activity-logs	Get activity logs
GET	/notifications	Get notifications
GET	/notifications/unread-count	Get unread count
PUT	/notifications/mark-all-read	Mark all notifications read
PUT	/notifications/{id}/read	Mark one notification read
POST	/walkguide/start	Log WalkGuide start
POST	/walkguide/stop	Log WalkGuide stop

Shared Call - /api/v1/shared/call

Method	Path	Description
POST	/token	Generate call token/channel payload
POST	/notify	Notify other party of incoming call
POST	/end	Notify/end call session

Design Patterns

The project documents and maps 7 GoF-related patterns in ooad-docs/.

#	Category	Pattern	Main Location
1	Creational	Builder	Backend entities/DTO construction, Lombok builders
2	Creational	Singleton	Flutter services registered through GetIt
3	Structural	Facade	Guardian dashboard aggregation and voice command/service coordination
4	Structural	Repository / Proxy	Spring Data repositories and Flutter repository-style data access
5	Behavioral	Observer	BLoC/Cubit state listeners and WebSocket callbacks
6	Behavioral	Strategy	Obstacle analysis / alert behavior mapping documented in OOAD
7	Behavioral	Chain of Responsibility	Spring Security filter chain and Dio interceptors

See:

• ooad-docs/DESIGN_PATTERNS.md
• ooad-docs/TRACEABILITY_AUDIT.md
• ooad-docs/01_Builder_Pattern.puml through ooad-docs/07_ChainOfResponsibility_Pattern.puml

Metrics And Evidence

The repository includes testing and benchmark support, but final scoring evidence should be generated on the target machine/device before submission.

Area	Current Support
Backend unit tests	JUnit/Mockito tests under src/test/java
Backend controller/integration tests	MockMvc and Testcontainers setup present
Coverage	JaCoCo configured in Maven
Load testing	k6 assets/results folder present under backend project
Flutter tests	Unit/widget/integration test files present
Flutter performance	Benchmark evidence template in ooad-docs/BENCHMARK_EVIDENCE_TEMPLATE.md

Recommended final evidence:

• mvn test or mvn verify output.
• JaCoCo HTML report.
• Testcontainers run on a Docker-enabled machine.
• k6 result at 50 or more concurrent virtual users.
• Flutter physical Android profile evidence: cold start, memory, jank/frame timing, CPU during AI, API latency, APK size.

Repository Structure

/
|-- walkguide-backend/
|   |-- demo/
|       |-- src/main/java/com/walkguide/
|       |   |-- config/
|       |   |-- controller/
|       |   |-- dto/
|       |   |-- entity/
|       |   |-- enums/
|       |   |-- exception/
|       |   |-- repository/
|       |   |-- security/
|       |   |-- service/
|       |   `-- websocket/
|       |-- src/main/resources/
|       |   |-- application.properties
|       |   |-- application-dev.yml
|       |   |-- application-prod.yml
|       |   `-- db/migration/V1...V17
|       |-- src/test/java/com/walkguide/
|       |-- k6-tests/
|       `-- pom.xml
|
|-- walkguide-mobile/
|   `-- walkguide_app/
|       |-- lib/
|       |   |-- app/
|       |   |-- core/
|       |   |-- features/
|       |   `-- shared/
|       |-- assets/
|       |   |-- images/
|       |   `-- models/
|       |-- test/
|       |-- integration_test/
|       `-- pubspec.yaml
|
|-- ooad-docs/
|   |-- 01_Builder_Pattern.puml
|   |-- 02_Singleton_Pattern.puml
|   |-- 03_Facade_Pattern.puml
|   |-- 04_Repository_Proxy_Pattern.puml
|   |-- 05_Observer_Pattern.puml
|   |-- 06_Strategy_Pattern.puml
|   |-- 07_ChainOfResponsibility_Pattern.puml
|   `-- diagrams/
|
|-- FULL_FLOW_ARCHITECTURE.md
|-- FINAL_EXAM_GUIDE.md
|-- TODO.md
`-- README.md

Feature Flows

Flow 1 - Register And Pairing

Guardian/User registers -> backend creates account and role data -> User receives pairing identity/code -> Guardian sends invite -> User responds -> backend activates pairing and seeds related configs.

Flow 2 - WalkGuide Detection

User starts WalkGuide -> camera/AI pipeline runs on device -> obstacle results are analyzed -> TTS/haptic feedback is emitted -> obstacle/location events can be logged to backend.

Flow 3 - SOS Alert

User triggers SOS -> backend creates SOS event -> Guardian receives/loads SOS event -> Guardian taps acknowledge or resolve -> SOS status no longer piles up as unhandled.

Flow 4 - Guardian Notification

Guardian opens send notification -> chooses text or voice note mode -> sends payload -> backend stores notification -> User notification screen can display/read/play supported message data.

Flow 5 - Location And Map

User sends location updates -> backend stores location history -> Guardian map/dashboard reads the latest location and history -> WebSocket support exists for real-time updates.

Flow 6 - Call

Caller requests token/channel -> backend returns call token payload -> caller notifies target -> target receives incoming call flow -> call can be ended through shared endpoint.

Quick Start

Backend

cd "D:\CodeSpace\Final Project Gabungan - Broken Test\walkguide-backend\demo"
.\mvnw.cmd spring-boot:run -Dspring-boot.run.profiles=dev

Health check:

http://localhost:8080/api/v1/auth/ping

Swagger UI:

http://localhost:8080/swagger-ui.html

Local/dev configuration reads database and secret values from environment variables or from the gitignored secrets.properties file:

$env:DB_URL="jdbc:postgresql://<host>:<port>/<database>"
$env:DB_USERNAME="<database_username>"
$env:DB_PASSWORD="<database_password>"
$env:JWT_SECRET="your-base64-secret"

Flutter

cd "D:\CodeSpace\Final Project Gabungan - Broken Test\walkguide-mobile\walkguide_app"
flutter pub get
flutter run

For Chrome/web debug:

flutter run -d chrome -t lib/main.dart

For Android APK:

flutter build apk --release

On a physical phone, the server URL must be reachable by the phone. Use the university server URL or your laptop LAN IP, not localhost.

Results

Final benchmark values should be filled from real test runs before submission.

Metric	Evidence Location / Tool	Current README Status
Backend tests	Maven/JUnit output	To be generated
Backend coverage	JaCoCo report	To be generated
Backend load	k6 results	Assets present, final run needed
Flutter tests	flutter test / integration tests	To be generated
Flutter performance	Physical Android profile evidence	To be generated
APK size	flutter build apk --analyze-size	To be generated

Weekly Progress

Week	Target	Current Status
1	Topic proposal, use case definitions, repo setup	Done
2-3	OOAD diagrams and traceability	Done, docs present in ooad-docs/
4	Spring Boot auth, pairing, entities, migrations	Implemented
5	Location, SOS, notification, WebSocket, FCM, call support	Implemented with demo/service integrations
6	Backend unit/integration testing and coverage setup	Implemented, final run evidence needed
7	Flutter server connect, auth, WalkGuide/YOLO support	Implemented
8	Guardian dashboard, SOS, notifications, voice notes, settings	Implemented
9	Integration testing and benchmark evidence	Needs final evidence run
10	Report, demo video, final submission polish	In progress

Core Objectives

O1 Performance: Keep obstacle detection local/on-device where possible and use backend for persistence, pairing, configuration, and real-time coordination.

O2 Accessibility: Support voice/TTS, haptic feedback, large touch targets, and hardware shortcut flows for visually impaired users.

O3 Traceability: Keep implementation aligned with FULL_FLOW_ARCHITECTURE.md, FINAL_EXAM_GUIDE.md, and the PUML diagrams in ooad-docs/.

O4 Configurability: Let Guardian configure AI sensitivity, voice commands, hardware shortcuts, geofence, and User settings through dashboard flows.

License

Distributed under the MIT License.

Final Exam: Integrated Mobile Application Project Flutter x Spring Boot x Object-Oriented Analysis and Design