update readme
This commit is contained in:
Wowieee4
parent
feb8e78b0c
commit
c5d716248e
422
README.md
422
README.md
@ -23,12 +23,14 @@
|
||||
[](https://flutter.dev/)
|
||||
[](https://spring.io/projects/spring-boot)
|
||||
[](https://www.postgresql.org/)
|
||||
[](https://firebase.google.com/)
|
||||
[](https://www.agora.io/)
|
||||
[](LICENSE)
|
||||
|
||||
|
||||
|
||||
|
||||
[**System Architecture**](#system-architecture) · [**Implementations**](#implementations) · [**Results**](#results) · [**Weekly Progress**](#weekly-progress)
|
||||
[**System Architecture**](#system-architecture) · [**Tech Stack**](#tech-stack) · [**Implementations**](#implementations) · [**API Endpoints**](#api-endpoints) · [**Design Patterns**](#design-patterns) · [**Results**](#results) · [**Weekly Progress**](#weekly-progress)
|
||||
|
||||
</div>
|
||||
|
||||
@ -38,7 +40,9 @@
|
||||
|
||||
**Core Objective:** How can we build an ultra-low latency, accessible navigation system for visually impaired users while providing real-time oversight for their guardians?
|
||||
|
||||
This project implements a dual-interface mobile application. The system relies on **On-Device AI (TFLite)** to eliminate network latency during obstacle detection, paired with a robust **Spring Boot backend** for secure authentication and guardian-user pairing.
|
||||
This project implements a dual-interface mobile application. The system relies on **On-Device AI (TFLite / YOLOv8n)** to eliminate network latency during obstacle detection, paired with a robust **Spring Boot backend** for secure authentication, guardian-user pairing, real-time location tracking via WebSocket, and push notifications via Firebase FCM.
|
||||
|
||||
**Deployment:** Backend is deployed on a university server at `202.46.28.160`. The Flutter APK uses a dynamic server URL — no hardcoded addresses — allowing multi-device testing without rebuilding.
|
||||
|
||||
---
|
||||
|
||||
@ -46,11 +50,68 @@ This project implements a dual-interface mobile application. The system relies o
|
||||
|
||||
The study follows a strict three-pillar enterprise structure:
|
||||
|
||||
**Pillar 1 — OOAD (Object-Oriented Analysis & Design):** Comprehensive modeling using Use Case, Class, Sequence, and ERD diagrams. The codebase strictly implements ≥ 4 GoF Design Patterns (e.g., Builder, Singleton, Repository, Strategy).
|
||||
**Pillar 1 — OOAD (Object-Oriented Analysis & Design):** Comprehensive modeling using Use Case, Class, Sequence, and ERD diagrams. The codebase strictly implements ≥ 7 GoF Design Patterns (Builder, Singleton, Facade, Repository/Proxy, Observer, Strategy, Chain of Responsibility).
|
||||
|
||||
**Pillar 2 — Flutter (Mobile Frontend):** Implements Clean Architecture (Domain, Data, Presentation layers) with BLoC/Riverpod for state management. Uses `Dio` for secure HTTP communication.
|
||||
**Pillar 2 — Flutter (Mobile Frontend):** Implements Clean Architecture (Domain, Data, Presentation layers) with BLoC for state management. Uses `Dio` with interceptors for secure HTTP communication. Server URL is dynamically configured via `SharedPreferences` on first launch.
|
||||
|
||||
**Pillar 3 — Spring Boot (Backend API):** A layered architecture (Controller → Service → Repository) powered by Java 17+. Features JWT-based Role-Based Access Control (RBAC) and standardized `ApiResponse` envelopes.
|
||||
**Pillar 3 — Spring Boot (Backend API):** A layered architecture (Controller → Service → Repository) powered by Java 21. Features JWT-based Role-Based Access Control (RBAC), standardized `ApiResponse` envelopes, WebSocket (STOMP) for real-time data, and Firebase Admin SDK for push notifications.
|
||||
|
||||
---
|
||||
|
||||
## Tech Stack
|
||||
|
||||
### Backend (Spring Boot)
|
||||
|
||||
| Library / Tool | Version | Purpose |
|
||||
|---|---|---|
|
||||
| Java | 21 | Primary language |
|
||||
| Spring Boot | 3.3.x | Main framework |
|
||||
| Spring Security | (bundled) | Auth + RBAC |
|
||||
| Spring Data JPA | (bundled) | ORM |
|
||||
| Spring WebSocket (STOMP) | (bundled) | Real-time location & notification push |
|
||||
| PostgreSQL Driver | (bundled) | DB connection — university server `202.46.28.160` |
|
||||
| Flyway | 10.x | Database schema migration |
|
||||
| JJWT | 0.11.5 | JWT access + refresh token |
|
||||
| Firebase Admin SDK | 9.x | FCM push notifications |
|
||||
| Agora RESTful API | - | Generate Agora RTC token for VoIP |
|
||||
| Springdoc OpenAPI | 2.3.0 | Swagger UI documentation |
|
||||
| Lombok | latest | Boilerplate reduction |
|
||||
| JUnit 5 + Mockito | (bundled) | Unit testing |
|
||||
| MockMvc + Testcontainers | 1.19.x | Integration testing with real PostgreSQL |
|
||||
| JaCoCo | 0.8.x | Code coverage (target ≥ 70%) |
|
||||
|
||||
### Flutter (Mobile)
|
||||
|
||||
| Package | Version | Purpose |
|
||||
|---|---|---|
|
||||
| flutter_bloc | 8.x | State management (sole pattern) |
|
||||
| go_router | 14.x | Navigation + role-based route guards |
|
||||
| dio | 5.x | HTTP client with interceptors |
|
||||
| shared_preferences | 2.x | Persist dynamic server URL |
|
||||
| flutter_secure_storage | 10.x | Secure JWT token storage |
|
||||
| drift | 2.x | SQLite ORM for offline cache |
|
||||
| tflite_flutter | 0.10.x | Run YOLOv8n on-device |
|
||||
| camera | 0.10.x | Camera feed for YOLO inference |
|
||||
| flutter_tts | 4.x | Text-to-Speech (ID + EN) |
|
||||
| speech_to_text | 6.x | Always-listening voice commands |
|
||||
| agora_rtc_engine | 6.x | VoIP call Guardian ↔ User |
|
||||
| firebase_messaging | 14.x | FCM push notification receiver |
|
||||
| flutter_map | 6.x | OpenStreetMap (free, no API key) |
|
||||
| geolocator | 11.x | Real-time GPS |
|
||||
| stomp_dart_client | 2.x | WebSocket STOMP for live tracking |
|
||||
| get_it | 7.x | Service locator / dependency injection |
|
||||
| dartz | 0.10.x | `Either<Failure, Data>` typed error handling |
|
||||
| vibration | 1.x | Haptic feedback on obstacle detection |
|
||||
|
||||
### External Services
|
||||
|
||||
| Service | Purpose | Cost |
|
||||
|---|---|---|
|
||||
| Firebase (FCM) | Push notifications | Free |
|
||||
| Agora RTC | In-app VoIP audio call | 10,000 min/month free |
|
||||
| University Server PostgreSQL | DB at `202.46.28.160:2002` | Free (managed by lecturer) |
|
||||
| OpenStreetMap + OSRM | Map tiles + turn-by-turn routing | Free |
|
||||
| YOLOv8n (Ultralytics) | Obstacle detection model (.tflite) | Free (open source) |
|
||||
|
||||
---
|
||||
|
||||
@ -58,65 +119,267 @@ The study follows a strict three-pillar enterprise structure:
|
||||
|
||||
### Design A — User Mode (Visually Impaired)
|
||||
An accessibility-first interface that opens directly to an active camera view.
|
||||
- **On-Device AI:** Obstacle detection using TFLite runs locally to ensure zero network latency.
|
||||
- **Hardware Mapping:** Physical volume buttons mapped to critical actions (e.g., auto-accept guardian calls).
|
||||
- **Voice Commands:** Integrated TTS and speech recognition for hands-free navigation.
|
||||
- **On-Device AI:** YOLOv8n obstacle detection via TFLite runs locally — zero network latency. Detects 80 COCO object classes, reports direction (LEFT/CENTER/RIGHT) and estimated distance.
|
||||
- **Voice Commands:** 14 configurable voice commands (e.g., "Start Walkguide", "Call Guardian", "Send SOS", "Where Am I"). Always-listening `SpeechToText` with auto-restart.
|
||||
- **Hardware Mapping:** Physical volume buttons mapped to critical actions (Vol Up = Call Guardian, Vol Down = Start WalkGuide by default).
|
||||
- **TTS Feedback:** Queued and immediate Text-to-Speech for obstacle alerts, navigation, and screen announcements. Supports Bahasa Indonesia and English.
|
||||
- **SOS System:** One-command emergency alert triggers high-priority FCM to Guardian with GPS coordinates.
|
||||
|
||||
### Design B — Guardian Mode (Admin)
|
||||
A command center dashboard for oversight and configuration.
|
||||
- **Real-time Monitoring:** Tracks user status and device battery.
|
||||
- **Remote Configuration:** Adjusts AI sensitivity and sets hardware shortcuts for the paired User device.
|
||||
- **Quick Actions:** One-tap voice conferencing and emergency overrides.
|
||||
### Design B — Guardian Mode (Admin/Caregiver)
|
||||
A command center dashboard for oversight and remote configuration.
|
||||
- **Real-time Live Map:** Tracks User's GPS location via WebSocket STOMP subscription, rendered on OpenStreetMap with `flutter_map`. Includes geofence circle overlay.
|
||||
- **Remote AI Configuration:** Adjusts YOLO confidence threshold, alert distance thresholds, max inference FPS, and enabled object labels.
|
||||
- **Voice Command Management:** Edits trigger phrases and enables/disables any of the 14 voice commands remotely.
|
||||
- **Notifications:** Sends text messages or voice notes (recorded audio) directly to the User's device.
|
||||
- **Geofence:** Sets a geographic boundary — backend notifies Guardian via FCM when User exits.
|
||||
|
||||
---
|
||||
|
||||
## Database Schema
|
||||
|
||||
Database runs on PostgreSQL at `202.46.28.160:2002`. Schema is managed exclusively by Flyway migrations. `spring.jpa.hibernate.ddl-auto=validate`.
|
||||
|
||||
| Migration | Table | Status |
|
||||
|---|---|---|
|
||||
| V1 | `users` | ✅ Exists |
|
||||
| V2 | Seed data | ✅ Exists |
|
||||
| V3 | Guardian-User link | ✅ Exists |
|
||||
| V4 | Add `unique_user_id` column | 🔄 Needs creation |
|
||||
| V5 | `pairing_relations` | 🔄 Needs creation |
|
||||
| V6 | `activity_logs` | 🔄 Needs creation |
|
||||
| V7 | `obstacle_logs` | 🔄 Needs creation |
|
||||
| V8 | `location_history` | 🔄 Needs creation |
|
||||
| V9 | `guardian_notifications` | 🔄 Needs creation |
|
||||
| V10 | `sos_events` | 🔄 Needs creation |
|
||||
| V11 | `user_settings` | 🔄 Needs creation |
|
||||
| V12 | `ai_configs` | 🔄 Needs creation |
|
||||
| V13 | `voice_command_configs` | 🔄 Needs creation |
|
||||
| V14 | `hardware_shortcuts` | 🔄 Needs creation |
|
||||
| V15 | `geofence_configs` | 🔄 Needs creation |
|
||||
| V16 | `refresh_tokens` | 🔄 Needs creation |
|
||||
|
||||
---
|
||||
|
||||
## API Endpoints
|
||||
|
||||
26 REST endpoints across 5 groups (exam minimum: 10).
|
||||
|
||||
**Auth** — `/api/v1/auth`
|
||||
|
||||
| Method | Path | Auth | Description |
|
||||
|---|---|---|---|
|
||||
| GET | `/ping` | None | Health check / server connection test |
|
||||
| POST | `/register` | None | Register Guardian or User |
|
||||
| POST | `/login` | None | Login, returns access + refresh token |
|
||||
| POST | `/refresh` | None | Refresh access token |
|
||||
| POST | `/logout` | ✅ | Logout, invalidate refresh token |
|
||||
| PUT | `/fcm-token` | ✅ | Update FCM device token |
|
||||
|
||||
**Pairing** — `/api/v1/shared/pairing`
|
||||
|
||||
| Method | Path | Auth | Description |
|
||||
|---|---|---|---|
|
||||
| POST | `/invite` | GUARDIAN | Invite User by 12-char unique ID |
|
||||
| POST | `/respond` | USER | Accept or reject pairing invite |
|
||||
| DELETE | `/unpair` | ✅ | Dissolve active pairing |
|
||||
| GET | `/status` | ✅ | Get current pairing status |
|
||||
|
||||
**Guardian** — `/api/v1/guardian`
|
||||
|
||||
| Method | Path | Description |
|
||||
|---|---|---|
|
||||
| GET | `/dashboard` | Combined home data (user status, recent logs, SOS count) |
|
||||
| GET | `/user-status` | Full status of paired User |
|
||||
| GET | `/user-location` | Last known GPS location |
|
||||
| GET | `/location-history` | Paginated location history |
|
||||
| GET | `/activity-logs` | Paginated activity logs of paired User |
|
||||
| GET | `/obstacle-logs` | Paginated obstacle detection logs |
|
||||
| POST | `/notifications/send` | Send text or voice note to User |
|
||||
| GET | `/sos-events` | Paginated SOS events |
|
||||
| PUT | `/sos/{id}/acknowledge` | Acknowledge SOS alert |
|
||||
| GET/PUT | `/ai-config` | Get or update AI configuration |
|
||||
| GET/PUT | `/voice-commands` | Get or update voice command configs |
|
||||
| GET/PUT | `/shortcuts` | Get or update hardware shortcut configs |
|
||||
| GET/PUT | `/geofence` | Get or update geofence config |
|
||||
|
||||
**User** — `/api/v1/user`
|
||||
|
||||
| Method | Path | Description |
|
||||
|---|---|---|
|
||||
| GET | `/profile` | User profile (id, email, displayName, uniqueUserId) |
|
||||
| GET/PUT | `/settings` | Get or update TTS/haptic settings |
|
||||
| GET | `/voice-commands` | Get all voice commands (read-only) |
|
||||
| GET/PUT | `/shortcuts` | Get or capture hardware shortcut assignments |
|
||||
| GET | `/ai-config` | Get AI config (read-only) |
|
||||
| POST | `/location` | Send GPS update |
|
||||
| POST | `/obstacle` | Log detected obstacle |
|
||||
| POST | `/sos` | Trigger SOS alert |
|
||||
| GET | `/activity-logs` | Paginated own activity log |
|
||||
| GET | `/notifications` | Paginated notifications from Guardian |
|
||||
| GET | `/notifications/unread-count` | Unread notification count |
|
||||
| PUT | `/notifications/mark-all-read` | Mark all as read |
|
||||
| PUT | `/notifications/{id}/read` | Mark one as read |
|
||||
| POST | `/walkguide/start` | Log WalkGuide session start |
|
||||
| POST | `/walkguide/stop` | Log WalkGuide session stop |
|
||||
|
||||
**Shared Call** — `/api/v1/shared/call`
|
||||
|
||||
| Method | Path | Description |
|
||||
|---|---|---|
|
||||
| POST | `/token` | Generate Agora RTC token |
|
||||
| POST | `/notify` | Send "Incoming Call" FCM to other party |
|
||||
|
||||
---
|
||||
|
||||
## Design Patterns
|
||||
|
||||
7 GoF Design Patterns implemented (exam minimum: 4, minimum 1 per category).
|
||||
|
||||
| # | Category | Pattern | Location |
|
||||
|---|---|---|---|
|
||||
| 1 | Creational | **Builder** | `User.java` (`@Builder`), `FcmService` message construction |
|
||||
| 2 | Creational | **Singleton** | Flutter: `TtsService`, `YoloDetector`, `WebSocketService`, `AgoraService` via GetIt |
|
||||
| 3 | Structural | **Facade** | Flutter: `VoiceCommandHandler`; Backend: `GuardianDashboardService` |
|
||||
| 4 | Structural | **Repository (Proxy)** | All `*_repository_impl.dart` — proxy between domain and data sources |
|
||||
| 5 | Behavioral | **Observer** | BLoC pattern (BLoC = Subject, Widgets = Observers); WebSocket callbacks |
|
||||
| 6 | Behavioral | **Strategy** | `ObstacleAnalyzer` direction strategy; `ObstacleAlertStrategyService` (backend) |
|
||||
| 7 | Behavioral | **Chain of Responsibility** | Spring Security filter chain; Dio interceptor chain |
|
||||
|
||||
---
|
||||
|
||||
## Metrics
|
||||
|
||||
| Metric | Instrument | Target Threshold |
|
||||
|--------|-----------|------|
|
||||
| Metric | Instrument | Target |
|
||||
|---|---|---|
|
||||
| API Throughput | Apache JMeter / k6 | ≥ 100 req/s under load |
|
||||
| API p95 Latency | Apache JMeter / k6 | < 500ms |
|
||||
| UI Frame Rate | Flutter DevTools | ≥ 90% frames < 16ms |
|
||||
| Code Coverage | JaCoCo | ≥ 70% (Service & Controller) |
|
||||
|
||||
---
|
||||
|
||||
## Technical Parameters
|
||||
|
||||
**Backend Phase (Spring Boot):**
|
||||
- Environment: Java 17+, PostgreSQL / H2 Database, Flyway Migrations.
|
||||
- Security: Spring Security, JWT (Access + Refresh Tokens).
|
||||
- Documentation: OpenAPI 3.0 (Swagger UI).
|
||||
|
||||
**Mobile Phase (Flutter):**
|
||||
- Environment: Flutter 3.x, physical Android device testing (Profile Mode).
|
||||
- Storage: `flutter_secure_storage` for JWT.
|
||||
- Routing: GoRouter with role-based authenticated routes.
|
||||
| APK Size | `flutter build apk --analyze-size` | < 50MB |
|
||||
|
||||
---
|
||||
|
||||
## Repository Structure
|
||||
|
||||
> 🚨 **DISCLAIMER:** This repository is in active development. The directory tree below is a **conceptual reference snapshot** of the intended architecture. It **does not** represent the exact, current state of the files.
|
||||
> 🚨 **DISCLAIMER:** This repository is in active development. The directory tree below reflects the intended final architecture. Flutter implementation is currently pending — backend (Spring Boot) is the active development phase.
|
||||
|
||||
```text
|
||||
/
|
||||
├── walkguide-backend/ # [Spring Boot Engine]
|
||||
├── walkguide-backend/ # [Spring Boot Engine — ACTIVE]
|
||||
│ ├── src/main/java/com/walkguide/
|
||||
│ │ ├── config/ # Security & Swagger configs
|
||||
│ │ ├── controller/ # REST Endpoints
|
||||
│ │ ├── dto/ # AuthRequest & ApiResponse
|
||||
│ │ ├── entity/ # JPA Entities
|
||||
│ │ └── exception/ # GlobalExceptionHandler
|
||||
│ └── pom.xml
|
||||
│ │ ├── config/ # SecurityConfig, WebSocketConfig, OpenApiConfig, FcmConfig, DataSeeder
|
||||
│ │ ├── enums/ # UserRole, PairingStatus, ActivityLogType, VoiceCommandKey,
|
||||
│ │ │ # HardwareShortcutKey, NotificationType, SosStatus
|
||||
│ │ ├── entity/ # User, PairingRelation, ActivityLog, ObstacleLog,
|
||||
│ │ │ # LocationHistory, GuardianNotification, SosEvent,
|
||||
│ │ │ # UserSettings, AiConfig, VoiceCommandConfig,
|
||||
│ │ │ # HardwareShortcut, GeofenceConfig, RefreshToken
|
||||
│ │ ├── repository/ # JPA repositories for all entities
|
||||
│ │ ├── dto/
|
||||
│ │ │ ├── request/ # RegisterRequest, LoginRequest, InviteUserRequest,
|
||||
│ │ │ │ # LocationUpdateRequest, ObstacleLogRequest,
|
||||
│ │ │ │ # SendNotificationRequest, SosRequest,
|
||||
│ │ │ │ # AiConfigUpdateRequest, VoiceCommandUpdateRequest,
|
||||
│ │ │ │ # HardwareShortcutUpdateRequest, GeofenceConfigRequest,
|
||||
│ │ │ │ # UserSettingsUpdateRequest, and more
|
||||
│ │ │ └── response/ # ApiResponse (standard wrapper), AuthDataResponse,
|
||||
│ │ │ # UserProfileResponse, PairingStatusResponse,
|
||||
│ │ │ # ActivityLogResponse, ObstacleLogResponse,
|
||||
│ │ │ # LocationResponse, NotificationResponse,
|
||||
│ │ │ # SosEventResponse, AgoraTokenResponse,
|
||||
│ │ │ # AiConfigResponse, VoiceCommandResponse,
|
||||
│ │ │ # HardwareShortcutResponse, GeofenceResponse
|
||||
│ │ ├── service/ # AuthService, PairingService, ActivityLogService,
|
||||
│ │ │ # LocationService, ObstacleLogService,
|
||||
│ │ │ # NotificationService, SosService,
|
||||
│ │ │ # AiConfigService, VoiceCommandService,
|
||||
│ │ │ # HardwareShortcutService, GeofenceService,
|
||||
│ │ │ # UserSettingsService, FcmService,
|
||||
│ │ │ # AgoraTokenService, GuardianDashboardService
|
||||
│ │ ├── controller/ # AuthController, PairingController,
|
||||
│ │ │ # GuardianController, UserController, CallController
|
||||
│ │ ├── security/ # JwtUtil, JwtAuthFilter, CustomUserDetailsService
|
||||
│ │ ├── websocket/ # LocationBroadcaster (STOMP broadcaster)
|
||||
│ │ └── exception/ # GlobalExceptionHandler, ResourceNotFoundException,
|
||||
│ │ # UnauthorizedException, PairingException
|
||||
│ ├── src/main/resources/
|
||||
│ │ ├── application.properties # DB config (PostgreSQL 202.46.28.160:2002), Flyway, JWT
|
||||
│ │ ├── firebase/
|
||||
│ │ │ └── google-services-admin.json # FCM service account key (gitignored!)
|
||||
│ │ └── db/migration/
|
||||
│ │ ├── V1__create_users_table.sql ✅ Exists
|
||||
│ │ ├── V2__seed_users.sql ✅ Exists
|
||||
│ │ ├── V3__link_guardian_user.sql ✅ Exists
|
||||
│ │ ├── V4__add_unique_user_id.sql 🔄 Needs creation
|
||||
│ │ ├── V5__create_pairing_relations.sql 🔄 Needs creation
|
||||
│ │ ├── V6__create_activity_logs.sql 🔄 Needs creation
|
||||
│ │ ├── V7__create_obstacle_logs.sql 🔄 Needs creation
|
||||
│ │ ├── V8__create_location_history.sql 🔄 Needs creation
|
||||
│ │ ├── V9__create_notifications.sql 🔄 Needs creation
|
||||
│ │ ├── V10__create_sos_events.sql 🔄 Needs creation
|
||||
│ │ ├── V11__create_user_settings.sql 🔄 Needs creation
|
||||
│ │ ├── V12__create_ai_configs.sql 🔄 Needs creation
|
||||
│ │ ├── V13__create_voice_commands.sql 🔄 Needs creation
|
||||
│ │ ├── V14__create_hardware_shortcuts.sql 🔄 Needs creation
|
||||
│ │ ├── V15__create_geofence_configs.sql 🔄 Needs creation
|
||||
│ │ └── V16__create_refresh_tokens.sql 🔄 Needs creation
|
||||
│ └── src/test/java/com/walkguide/
|
||||
│ ├── service/ # AuthServiceTest, PairingServiceTest,
|
||||
│ │ # NotificationServiceTest, SosServiceTest,
|
||||
│ │ # LocationServiceTest, AiConfigServiceTest
|
||||
│ └── controller/ # AuthControllerTest, GuardianControllerTest,
|
||||
│ # UserControllerTest (MockMvc + Testcontainers)
|
||||
│
|
||||
├── walkguide-mobile/ # [Flutter Frontend]
|
||||
├── walkguide-mobile/ # [Flutter Frontend — PENDING]
|
||||
│ ├── lib/
|
||||
│ │ ├── core/ # API Service, Storage, Failures
|
||||
│ │ ├── main.dart # Firebase init, GetIt setup, camera init
|
||||
│ │ ├── app/
|
||||
│ │ │ ├── app.dart # MultiBlocProvider + MaterialApp.router
|
||||
│ │ │ ├── router.dart # GoRouter: /server-connect → /splash → role-based routes
|
||||
│ │ │ └── injection_container.dart # GetIt singletons & factories
|
||||
│ │ ├── core/
|
||||
│ │ │ ├── constants/ # app_constants.dart (dynamic BASE_URL via SharedPreferences),
|
||||
│ │ │ │ # route_constants.dart, voice_command_keys.dart
|
||||
│ │ │ ├── errors/ # failures.dart, exceptions.dart
|
||||
│ │ │ ├── network/ # api_client.dart (Dio), auth_interceptor.dart,
|
||||
│ │ │ │ # error_interceptor.dart, network_info.dart
|
||||
│ │ │ ├── storage/ # secure_storage.dart (JWT), local_database.dart (Drift)
|
||||
│ │ │ ├── services/ # tts_service.dart, stt_service.dart,
|
||||
│ │ │ │ # voice_command_handler.dart,
|
||||
│ │ │ │ # hardware_shortcut_listener.dart,
|
||||
│ │ │ │ # fcm_service.dart, haptic_service.dart,
|
||||
│ │ │ │ # websocket_service.dart, agora_service.dart
|
||||
│ │ │ ├── ai/ # model_loader.dart, yolo_detector.dart,
|
||||
│ │ │ │ # obstacle_analyzer.dart
|
||||
│ │ │ ├── theme/ # app_theme.dart, app_colors.dart
|
||||
│ │ │ └── utils/ # permission_manager.dart, location_service.dart
|
||||
│ │ ├── features/
|
||||
│ │ │ ├── auth/ # Clean Arch: domain, data, presentation
|
||||
│ │ │ └── home/ # User & Guardian Dashboards
|
||||
│ │ └── main.dart
|
||||
│ │ │ ├── server_connect/ # ServerConnectScreen — FIRST SCREEN on fresh install
|
||||
│ │ │ │ # BLoC: TestConnection, SaveAndContinue, ChangeServer
|
||||
│ │ │ ├── auth/ # SplashScreen, LoginScreen, RegisterScreen
|
||||
│ │ │ │ # Clean Arch: domain / data / presentation
|
||||
│ │ │ ├── pairing/ # UserPairingScreen (show unique ID, accept invite)
|
||||
│ │ │ │ # GuardianPairingScreen (invite by 12-char ID)
|
||||
│ │ │ ├── walk_guide/ # WalkGuideScreen (full-screen camera + YOLO overlay)
|
||||
│ │ │ │ # WalkGuideBloc: Start → camera stream → detect → TTS
|
||||
│ │ │ ├── sos/ # SosScreen (large SOS button)
|
||||
│ │ │ ├── activity_log/ # ActivityLogScreen (paginated, filterable)
|
||||
│ │ │ ├── notifications/ # NotificationScreen + TTS read-all feature
|
||||
│ │ │ ├── navigation_mode/ # NavigationModeScreen (OSM map + OSRM routing)
|
||||
│ │ │ ├── settings/ # UserSettingsScreen (TTS, pairing, account)
|
||||
│ │ │ ├── call/ # CallScreen + IncomingCallScreen (Agora VoIP)
|
||||
│ │ │ ├── manual/ # ManualScreen (all voice commands & shortcuts)
|
||||
│ │ │ └── guardian_dashboard/ # GuardianDashboardScreen, GuardianMapScreen,
|
||||
│ │ │ # GuardianActivityLogScreen, GuardianSendNotifScreen,
|
||||
│ │ │ # GuardianAiConfigScreen, GuardianVoiceCmdScreen,
|
||||
│ │ │ # GuardianShortcutScreen, GuardianGeofenceScreen
|
||||
│ │ └── shared/widgets/ # AppBottomNav, VoiceCommandListener,
|
||||
│ │ # TtsAwareScaffold, LoadingOverlay, ErrorWidget
|
||||
│ ├── assets/
|
||||
│ │ ├── models/
|
||||
│ │ │ ├── yolov8n.tflite # YOLOv8n converted to TFLite (~6MB)
|
||||
│ │ │ └── labels.txt # 80 COCO object labels
|
||||
│ │ └── images/
|
||||
│ └── pubspec.yaml
|
||||
│
|
||||
├── ooad-docs/ # [Design Artifacts]
|
||||
@ -124,68 +387,103 @@ A command center dashboard for oversight and configuration.
|
||||
│ └── Traceability_Matrix.md
|
||||
│
|
||||
└── README.md
|
||||
````
|
||||
```
|
||||
|
||||
-----
|
||||
---
|
||||
|
||||
## Feature Flows (High-Level)
|
||||
|
||||
### Flow 1 — Register & Pairing
|
||||
Guardian registers → User registers (gets a 12-char `uniqueUserId`) → Guardian enters User's ID → backend sends FCM invite → User accepts in app → backend seeds 14 default voice commands, 5 hardware shortcuts, and AI config for the pair.
|
||||
|
||||
### Flow 2 — WalkGuide Active Detection
|
||||
User says "Start Walkguide" (or presses Volume Down) → camera starts → frames are throttled to max inference FPS → YOLOv8n detects obstacles → direction and distance are analyzed → TTS announces "Caution! Person ahead center. Very close. Please stop." → haptic feedback triggers → obstacle logged to backend → GPS location broadcast via WebSocket to Guardian's live map every 5 seconds.
|
||||
|
||||
### Flow 3 — VoIP Call
|
||||
User says "Call Guardian" → Agora token fetched from backend → FCM "Incoming Call" sent to Guardian → Guardian accepts in `IncomingCallScreen` → both join same Agora RTC channel → audio call connects.
|
||||
|
||||
### Flow 4 — Guardian Sends Notification → User Hears via TTS
|
||||
Guardian sends text/voice note → backend saves notification + sends FCM → User receives in foreground via WebSocket → `flutter_local_notifications` shown + TTS announced → User says "Read All My Notifications" → TTS reads each message → marked as read.
|
||||
|
||||
### Flow 5 — SOS Alert
|
||||
User says "Send SOS" (or hardware shortcut) → GPS captured → backend saves SOS event → high-priority FCM sent to Guardian → Guardian map opens to User's location → Guardian taps Acknowledge → FCM sent back to User → TTS "Guardian is on the way."
|
||||
|
||||
### Flow 6 — Geofence
|
||||
Guardian sets center + radius on map → User's location updates checked against Haversine distance → on exit: FCM to Guardian + `GEOFENCE_EXIT` activity log.
|
||||
|
||||
---
|
||||
|
||||
## Quick Start (WIP)
|
||||
|
||||
```bash
|
||||
# 1. Clone the repository
|
||||
git clone [https://github.com/YourGroup/walkguide-final-exam.git](https://github.com/YourGroup/walkguide-final-exam.git)
|
||||
git clone https://github.com/YourGroup/walkguide-final-exam.git
|
||||
|
||||
# 2. Run Backend (Spring Boot)
|
||||
cd walkguide-backend
|
||||
./mvnw spring-boot:run
|
||||
# Flyway auto-migrates V1–V16 on first start
|
||||
# Swagger UI: http://202.46.28.160:8080/swagger-ui.html
|
||||
# Health check: GET http://202.46.28.160:8080/api/v1/auth/ping
|
||||
|
||||
# 3. Run Mobile (Flutter) - Connect to local backend
|
||||
# 3. Run Mobile (Flutter) - Connect to local or university backend
|
||||
cd ../walkguide-mobile
|
||||
flutter pub get
|
||||
flutter run
|
||||
# On first launch: enter server URL in ServerConnectScreen
|
||||
# Example: http://202.46.28.160:8080
|
||||
```
|
||||
|
||||
-----
|
||||
---
|
||||
|
||||
## Results
|
||||
|
||||
> ⏳ **Work In Progress:** Results are populated as benchmarking phases are completed.
|
||||
|
||||
| Metric | Baseline | Final Optimized | Status |
|
||||
|--------|-----------------|----------------------|---------|
|
||||
|---|---|---|---|
|
||||
| Cold Start Time | *Pending* | *Pending* | ⏳ |
|
||||
| Memory Leak (10 Navs)| *Pending* | *Pending* | ⏳ |
|
||||
| Memory Leak (10 Navs) | *Pending* | *Pending* | ⏳ |
|
||||
| API Error Rate | *Pending* | *Pending* | ⏳ |
|
||||
| YOLO Inference Latency (ms) | *Pending* | *Pending* | ⏳ |
|
||||
| API p95 Latency | *Pending* | *Pending* | ⏳ |
|
||||
| JaCoCo Coverage | *Pending* | *Pending* | ⏳ |
|
||||
|
||||
-----
|
||||
---
|
||||
|
||||
## Weekly Progress
|
||||
|
||||
| Week | Target | Status |
|
||||
|------|--------|--------|
|
||||
|---|---|---|
|
||||
| 1 | Topic proposal, Use Case definitions, Repo setup | ✅ Done |
|
||||
| 2-3 | OOAD diagrams, OpenAPI YAML drafted | 🔄 In Progress |
|
||||
| 4 | Clean Architecture & Spring Boot layers implemented | ⏳ Pending |
|
||||
| 5 | Mid-sprint benchmarking (Flutter + Backend) | ⏳ Pending |
|
||||
| 6-7 | Feature freeze, AI integration, Integration testing | ⏳ Pending |
|
||||
| 8 | Final benchmarks, Report writing, Demo Video | ⏳ Pending |
|
||||
| 2–3 | OOAD diagrams, OpenAPI YAML drafted | 🔄 In Progress |
|
||||
| 4 | Spring Boot: Auth, Pairing, Entity, Migration V4–V16 | 🔄 In Progress |
|
||||
| 5 | Spring Boot: Location, SOS, Notification, WS, FCM, Agora | ⏳ Pending |
|
||||
| 6 | Spring Boot: Unit + Integration tests, JaCoCo ≥ 70% | ⏳ Pending |
|
||||
| 7 | Flutter: ServerConnect, Auth, WalkGuide + YOLO pipeline | ⏳ Pending |
|
||||
| 8 | Flutter: Guardian dashboard, Call, SOS, Notifications | ⏳ Pending |
|
||||
| 9 | Feature freeze, integration testing, benchmark on device | ⏳ Pending |
|
||||
| 10 | Final benchmarks, Report writing, Demo Video | ⏳ Pending |
|
||||
|
||||
-----
|
||||
---
|
||||
|
||||
## Core Objectives
|
||||
|
||||
**O₁ (Performance):** The dual-platform architecture must maintain a 60fps UI frame rate while handling concurrent backend requests without exceeding 500ms latency.
|
||||
**O₁ (Performance):** The dual-platform architecture must maintain a 60fps UI frame rate while handling concurrent backend requests without exceeding 500ms latency. YOLO inference runs on a separate isolate to avoid blocking the UI thread.
|
||||
|
||||
**O₂ (Accessibility):** The User mode must require zero visual interaction post-login, relying entirely on haptics, voice, and physical button mapping.
|
||||
**O₂ (Accessibility):** The User mode must require zero visual interaction post-login, relying entirely on haptics, voice commands, and physical button mapping. All screen transitions are announced via TTS.
|
||||
|
||||
**O₃ (Traceability):** Every major feature implementation must be directly traceable back to the pre-development OOAD artifacts and GoF design patterns.
|
||||
|
||||
-----
|
||||
**O₄ (Configurability):** All AI sensitivity settings, voice command phrases, and hardware shortcuts are remotely configurable by the Guardian without requiring an app update.
|
||||
|
||||
---
|
||||
|
||||
## License
|
||||
|
||||
Distributed under the [MIT License](https://www.google.com/search?q=LICENSE).
|
||||
Distributed under the [MIT License](LICENSE).
|
||||
|
||||
-----
|
||||
---
|
||||
|
||||
*Final Exam: Integrated Mobile Application Project* <br>
|
||||
*Flutter × Spring Boot × Object-Oriented Analysis and Design*
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user