Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
1b1c000264 | ||
|
|
081c95365b | ||
|
|
ef7ecf9e51 | ||
|
|
6392f1c2c6 | ||
|
|
393ceadef1 | ||
|
|
0f351cb49e | ||
|
|
fe63a50997 | ||
|
|
e827aa0e96 | ||
|
|
24017fb437 | ||
|
|
0364638834 | ||
|
|
1e8e7e39dd | ||
|
|
8f8acbc24b | ||
|
|
3701a40b34 | ||
|
|
00a821e06d | ||
|
|
5ac0a227c4 | ||
|
|
7492704c8e | ||
|
|
fbde479958 | ||
|
|
e6e31080a1 |
@@ -0,0 +1,30 @@
|
||||
# Python & Bytecode
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
*$py.class
|
||||
*.so
|
||||
.Python
|
||||
build/
|
||||
dist/
|
||||
*.egg-info/
|
||||
|
||||
# Environments & Virtual Environments
|
||||
.venv/
|
||||
venv/
|
||||
ENV/
|
||||
env/
|
||||
.env
|
||||
|
||||
# uv & Version Pinning
|
||||
uv.lock
|
||||
.python-version
|
||||
|
||||
# IDEs & Notebooks
|
||||
.vscode/
|
||||
.idea/
|
||||
.ipynb_checkpoints/
|
||||
|
||||
# OS
|
||||
.DS_Store
|
||||
Thumbs.db
|
||||
|
||||
@@ -1,4 +0,0 @@
|
||||
venv/
|
||||
__pycache__/
|
||||
*.keras
|
||||
.vscode/
|
||||
@@ -0,0 +1,98 @@
|
||||
# Praktikum Deep Learning - Kelompok 3
|
||||
|
||||
Repository ini berisi implementasi tugas praktikum Deep Learning, mencakup:
|
||||
1. **Multi-Layer Perceptron (MLP)**: Eksperimen kedalaman layer & regularisasi (*Batch Normalization* & *Dropout*) pada dataset MNIST.
|
||||
2. **Model Selection**: Studi kasus pemilihan arsitektur model sesuai karakteristik data (Time Series, Computer Vision, NLP, dan Tabular).
|
||||
3. **Convolutional Neural Network (CNN)**: Pipeline klasifikasi citra MNIST, eksplorasi kedalaman arsitektur (2 blok vs 3 blok), serta analisis performa mendalam.
|
||||
4. **Natural Language Processing (NLP)**: Klasifikasi sentimen ulasan produk Tokopedia menggunakan arsitektur *Embedding* dan *GlobalAveragePooling1D*.
|
||||
|
||||
---
|
||||
|
||||
## 📁 Struktur Direktori
|
||||
|
||||
```text
|
||||
kelompok3-deeplearning/
|
||||
├── requirements.txt # Dependensi Python proyek
|
||||
├── README.md # Panduan utama proyek
|
||||
├── docs/ # Laporan & materi presentasi
|
||||
│ ├── laporan_akhir.md # Laporan akhir praktikum lengkap
|
||||
│ ├── PRESENTATION.md # Panduan materi slide presentasi
|
||||
│ ├── model_selection.md # Analisis justifikasi pemilihan model
|
||||
│ └── analisis_evaluasi_model_sentimen_gap.md # Analisis model NLP Tokopedia
|
||||
├── src/ # Modul bersama (data loader, arsitektur, utils)
|
||||
│ ├── data_loader.py # Data loader & preprocessor MNIST
|
||||
│ ├── models.py # Fungsi pembangun arsitektur model
|
||||
│ └── utils.py # Timer callback & helper visualisasi
|
||||
├── tasks/ # Skrip tugas yang siap dieksekusi per topik
|
||||
│ ├── mlp/ # Topik 1: Multi-Layer Perceptron
|
||||
│ │ ├── 01_mlp_comparison.py # Perbandingan 3 arsitektur MLP (1, 2, 3 hidden layer)
|
||||
│ │ ├── 02_regularization.py # Uji regularisasi BatchNorm + Dropout pada MLP
|
||||
│ │ └── 03_model_selection.py # Implementasi contoh arsitektur model selection
|
||||
│ ├── cnn/ # Topik 2: Convolutional Neural Network
|
||||
│ │ ├── 04_cnn_mnist.py # Baseline CNN MNIST & perbandingan awal vs MLP
|
||||
│ │ ├── 05_cnn_full_pipeline.py # Training pipeline CNN lengkap & confusion matrix
|
||||
│ │ ├── 06_cnn_deeper.py # Eksplorasi kedalaman arsitektur CNN (2 blok vs 3 blok)
|
||||
│ │ └── 07_cnn_evaluation.py # Evaluasi mendalam (metrik per kelas, confidence, error analysis)
|
||||
│ └── nlp_sentiment/ # Topik 3: NLP Analisis Sentimen
|
||||
│ └── 08_sentimen_gap.py # Klasifikasi sentimen ulasan Tokopedia (GAP Model)
|
||||
└── results/ # Output hasil training (grafik, bobot model, laporan)
|
||||
├── 01_mlp/ # Output tugas MLP
|
||||
├── 02_cnn/ # Output tugas CNN
|
||||
└── 03_nlp_sentiment/ # Output tugas NLP Sentimen
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Panduan Menjalankan
|
||||
|
||||
### 1. Setup Virtual Environment & Install Dependensi
|
||||
|
||||
```bash
|
||||
# Buat dan aktifkan virtual environment
|
||||
python3 -m venv .venv
|
||||
source .venv/bin/activate
|
||||
|
||||
# Install seluruh dependensi yang dibutuhkan
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
### 2. Eksekusi Script per Topik
|
||||
|
||||
#### 🔹 Topik 1: Multi-Layer Perceptron (MLP)
|
||||
```bash
|
||||
python tasks/mlp/01_mlp_comparison.py
|
||||
python tasks/mlp/02_regularization.py
|
||||
python tasks/mlp/03_model_selection.py
|
||||
```
|
||||
|
||||
#### 🔹 Topik 2: Convolutional Neural Network (CNN)
|
||||
```bash
|
||||
python tasks/cnn/04_cnn_mnist.py
|
||||
python tasks/cnn/05_cnn_full_pipeline.py
|
||||
python tasks/cnn/06_cnn_deeper.py
|
||||
python tasks/cnn/07_cnn_evaluation.py
|
||||
```
|
||||
|
||||
#### 🔹 Topik 3: NLP Analisis Sentimen (Tokopedia)
|
||||
```bash
|
||||
python tasks/nlp_sentiment/08_sentimen_gap.py
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📚 Dokumen Terkait
|
||||
|
||||
- **Laporan Akhir Praktikum:** [docs/laporan_akhir.md](docs/laporan_akhir.md)
|
||||
- **Materi Presentasi (PPT):** [docs/PRESENTATION.md](docs/PRESENTATION.md)
|
||||
- **Analisis Model Selection:** [docs/model_selection.md](docs/model_selection.md)
|
||||
- **Analisis Sentimen Tokopedia:** [docs/analisis_evaluasi_model_sentimen_gap.md](docs/analisis_evaluasi_model_sentimen_gap.md)
|
||||
|
||||
---
|
||||
|
||||
## 🛠️ Stack Teknologi
|
||||
|
||||
- **Bahasa:** Python 3.10+
|
||||
- **Deep Learning:** TensorFlow & Keras
|
||||
- **NLP & Dataset:** Hugging Face Datasets & Scikit-Learn
|
||||
- **Data & Visualisasi:** NumPy, Pandas, Matplotlib, & Seaborn
|
||||
- **Machine Learning Klasik:** XGBoost
|
||||
@@ -0,0 +1,136 @@
|
||||
# 📊 Presentasi Praktikum Deep Learning
|
||||
**Kelompok 3**
|
||||
|
||||
---
|
||||
|
||||
## 1. INTRODUKSI
|
||||
|
||||
### 📌 Latar Belakang
|
||||
Praktikum ini bertujuan untuk mengeksplorasi berbagai arsitektur jaringan saraf tiruan (Neural Network) dan teknik pelatihan, mulai dari Multi-Layer Perceptron (MLP), mitigasi overfitting melalui regularisasi, analisis pemilihan model (*Model Selection*), hingga Convolutional Neural Network (CNN) pada dataset MNIST dan domain data lainnya.
|
||||
|
||||
### 📦 Dataset Utama: MNIST Handwritten Digits
|
||||
* **Ukuran Dataset:** 60.000 citra training & 10.000 citra testing.
|
||||
* **Format Data:** Citra grayscale berukuran $28 \times 28$ piksel.
|
||||
* **Jumlah Kelas:** 10 kelas target (digit angka 0 sampai 9).
|
||||
* **Preprocessing:**
|
||||
* Normalisasi nilai piksel dari $[0, 255]$ menjadi rentang $[0.0, 1.0]$.
|
||||
* Penyesuaian bentuk data (vektor 784 untuk MLP, tensor $28 \times 28 \times 1$ untuk CNN).
|
||||
|
||||
---
|
||||
|
||||
## 2. PENJELASAN TUGAS 1: Eksperimen Arsitektur & Kedalaman MLP
|
||||
|
||||
### 🎯 Tujuan
|
||||
Menguji pengaruh kedalaman layer dan jumlah neuron terhadap akurasi model serta waktu pelatihan pada dataset MNIST.
|
||||
|
||||
### 🏗️ Konfigurasi 3 Model MLP
|
||||
* **Model A (1 Hidden Layer):** Input (784) $\rightarrow$ `Dense(64, ReLU)` $\rightarrow$ `Dense(10, Softmax)`
|
||||
* **Model B (2 Hidden Layer):** Input (784) $\rightarrow$ `Dense(128, ReLU)` $\rightarrow$ `Dense(64, ReLU)` $\rightarrow$ `Dense(10, Softmax)`
|
||||
* **Model C (3 Hidden Layer):** Input (784) $\rightarrow$ `Dense(256, ReLU)` $\rightarrow$ `Dense(128, ReLU)` $\rightarrow$ `Dense(64, ReLU)` $\rightarrow$ `Dense(10, Softmax)`
|
||||
|
||||
### 📈 Hasil Eksperimen
|
||||
| Model | Arsitektur Hidden Layer | Jumlah Parameter | Test Accuracy | Waktu Training / Epoch |
|
||||
| :--- | :--- | :---: | :---: | :---: |
|
||||
| **Model A** | `[64]` | 50.890 | 97,06% | 1,22 detik |
|
||||
| **Model B** | `[128, 64]` | 109.386 | 97,24% | 1,29 detik |
|
||||
| **Model C (Terbaik)** | `[256, 128, 64]` | 242.762 | **97,64%** | 2,50 detik |
|
||||
|
||||
### 💡 Kesimpulan Tugas 1
|
||||
* **Model C** menghasilkan akurasi tertinggi (**97,64%**) karena mampu mempelajari representasi fitur yang lebih kompleks.
|
||||
* Semakin dalam layer, jumlah parameter dan waktu komputasi per epoch meningkat secara signifikan.
|
||||
|
||||
---
|
||||
|
||||
## 3. PENJELASAN TUGAS 2: Eksperimen Regularisasi pada Deep MLP
|
||||
|
||||
### 🎯 Tujuan
|
||||
Mengatasi permasalahan *overfitting* yang muncul pada Model C saat dilatih dalam durasi yang lebih panjang (20 epoch).
|
||||
|
||||
### 🔍 Permasalahan pada Baseline (Tanpa Regularisasi)
|
||||
* *Training loss* terus menurun mendekati 0, namun *validation loss* mulai naik dan berfluktuasi setelah epoch ke-7.
|
||||
* Terjadi pelebaran *generalization gap* (model menghafal data training, bukan mempelajari pola umum).
|
||||
|
||||
### 🛡️ Solusi: Penerapan Batch Normalization & Dropout
|
||||
* **Batch Normalization:** Menormalkan input di setiap layer, menstabilkan distribusi gradien, dan mempercepat konvergensi.
|
||||
* **Dropout (rate = 0.3):** Mematikan 30% koneksi neuron secara acak selama training untuk mencegah *co-adaptation*.
|
||||
|
||||
```text
|
||||
Arsitektur Regularized:
|
||||
Input (784) ──> Dense(256) ──> BatchNorm ──> ReLU ──> Dropout(0.3)
|
||||
──> Dense(128) ──> BatchNorm ──> ReLU ──> Dropout(0.3)
|
||||
──> Dense(64) ──> BatchNorm ──> ReLU ──> Dropout(0.3)
|
||||
──> Dense(10, Softmax)
|
||||
```
|
||||
|
||||
### 💡 Kesimpulan Tugas 2
|
||||
* Penambahan BatchNorm dan Dropout berhasil menjaga kurva *validation loss* tetap stabil dan sejajar dengan *training loss*.
|
||||
* Daya generalisasi model meningkat dan risiko overfitting berhasil ditekan secara efektif.
|
||||
|
||||
---
|
||||
|
||||
## 4. PENJELASAN TUGAS 3: Analisis & Justifikasi Pemilihan Model (*Model Selection*)
|
||||
|
||||
### 🎯 Tujuan
|
||||
Menentukan dan menjustifikasi arsitektur model yang paling optimal berdasarkan karakteristik data input dan kebutuhan komputasi.
|
||||
|
||||
### 📋 Hasil Pemilihan Model untuk 4 Skenario:
|
||||
|
||||
#### (a) Prediksi Stok Barang dari 12 Bulan Historis Penjualan
|
||||
* **Tipe Data:** Deret Waktu (*Time-Series*).
|
||||
* **Arsitektur Terpilih:** **LSTM (Long Short-Term Memory)** *(Alternatif: XGBoost / SARIMA)*.
|
||||
* **Justifikasi:** Data memiliki dependensi urutan waktu dan tren musiman. LSTM mampu mempertahankan memori jangka panjang melalui *cell state* dan mekanisme *gates* tanpa terkena *vanishing gradient*.
|
||||
|
||||
#### (b) Klasifikasi Foto Produk Rusak vs Normal di Lini Produksi
|
||||
* **Tipe Data:** Citra 2D (*Computer Vision*).
|
||||
* **Arsitektur Terpilih:** **CNN (Convolutional Neural Network)**.
|
||||
* **Justifikasi:** Pola kerusakan (goresan/retakan) terikat pada korelasi spasial piksel. CNN mengekstrak hierarki fitur visual secara bertingkat dan memiliki sifat *translation invariance* (tahan pergeseran).
|
||||
|
||||
#### (c) Analisis Sentimen Review Google Play Store
|
||||
* **Tipe Data:** Teks Bebas Bahasa Alami (*NLP / Sequential Text*).
|
||||
* **Arsitektur Terpilih:** **LSTM dengan Word Embedding** *(Alternatif: Transformer / IndoBERT)*.
|
||||
* **Justifikasi:** Ulasan teks adalah sekuens kata di mana arti kalimat sangat bergantung pada urutan. Embedding memetakan makna semantik kata, dan LSTM memahami konteks kalimat dari awal hingga akhir.
|
||||
|
||||
#### (d) Prediksi Churn Pelanggan dari 15 Kolom Tabular
|
||||
* **Tipe Data:** Tabular Terstruktur (*Structured Tabular Data*).
|
||||
* **Arsitektur Terpilih:** **ML Klasik (XGBoost / LightGBM)**.
|
||||
* **Justifikasi:** Pada data tabular dimensi kecil (15 fitur), model *gradient boosted trees* secara konsisten mengungguli deep learning dalam performa generalisasi, kecepatan latih, dan interpretasi fitur (*feature importance*).
|
||||
|
||||
---
|
||||
|
||||
## 5. PENJELASAN TUGAS 4: Implementasi CNN pada MNIST vs MLP
|
||||
|
||||
### 🎯 Tujuan
|
||||
Membangun model CNN 2D sederhana dan membandingkan performanya secara kuantitatif dengan model MLP terbaik dari Tugas 1 (Model C).
|
||||
|
||||
### 🏗️ Arsitektur Model CNN
|
||||
```text
|
||||
Input (28, 28, 1)
|
||||
↓
|
||||
Conv2D (32 filter, 3×3, ReLU) ──> Output: (26, 26, 32)
|
||||
MaxPooling2D (2×2) ──> Output: (13, 13, 32)
|
||||
↓
|
||||
Conv2D (64 filter, 3×3, ReLU) ──> Output: (11, 11, 64)
|
||||
MaxPooling2D (2×2) ──> Output: (5, 5, 64)
|
||||
↓
|
||||
Flatten ──> Output: 1600
|
||||
Dense (64, ReLU) + Dropout (0.3)
|
||||
↓
|
||||
Dense (10, Softmax) ──> Output: Probabilitas 10 Kelas
|
||||
```
|
||||
|
||||
### 📊 Tabel Perbandingan Kuantitatif: CNN vs MLP Terbaik
|
||||
| Metrik Evaluasi | MLP Terbaik (Model C) | Model CNN Sederhana | Analisis / Improvement |
|
||||
| :--- | :---: | :---: | :--- |
|
||||
| **Test Accuracy** | 97,64% | **99,03%** | **+1,39 poin persentase** |
|
||||
| **Test Loss** | 0,0933 | **0,0297** | **Turun signifikan (-68,2%)** |
|
||||
| **Error Rate** | 2,36% | **0,97%** | **Pengurangan error 58,9% (relatif)** |
|
||||
| **Salah Klasifikasi** | 236 dari 10.000 | **97 dari 10.000** | **139 gambar lebih sedikit salah** |
|
||||
| **Jumlah Parameter** | 242.762 | **121.930** | **Hemat 49,8% parameter (separuh bobot)** |
|
||||
| **Waktu / Epoch (CPU)** | **2,50 detik** | 16,56 detik | Konvolusi membutuhkan komputasi lebih di CPU |
|
||||
|
||||
### 💡 Mengapa CNN Lebih Unggul untuk Data Citra?
|
||||
1. **Preservasi Struktur Spasial:** Memproses citra dalam format 2D asli sehingga korelasi antarpiksel tetangga tetap terjaga (tidak hilang seperti pada *flattening* MLP).
|
||||
2. **Parameter Sharing:** Filter konvolusi yang sama digeser ke seluruh permukaan citra untuk mendeteksi pola yang sama di berbagai posisi.
|
||||
3. **Hierarki Fitur Otomatis:** Layer awal mengekstrak garis tepi/sudut, sedangkan layer lanjutan menggabungkannya menjadi bentuk kurva dan pola angka utuh.
|
||||
4. **Translation Invariance:** Fitur `MaxPooling2D` membuat prediksi model tetap konsisten meskipun posisi tulisan tangan sedikit bergeser.
|
||||
|
||||
@@ -0,0 +1,68 @@
|
||||
# Analisis Sentimen Lengkap (GAP Model)
|
||||
|
||||
## 1. Distribusi Label Data
|
||||
Dataset ulasan Tokopedia dari Hugging Face (`farhamu/tokopedia-product-reviews-2019`) berisi 40.607 baris ulasan. Setelah membersihkan rating netral (rating 3 sebanyak 1.823 baris), tersisa **38.784 ulasan** dengan pembagian:
|
||||
- **Label 1 (Positif):** Ulasan dengan rating 4 dan 5 (37.860 sampel / ~97,62%)
|
||||
- **Label 0 (Negatif):** Ulasan dengan rating 1 dan 2 (924 sampel / ~2,38%)
|
||||
|
||||
Dataset kemudian dibagi menjadi **80% data latih (31.027 ulasan)** dan **20% data uji (7.757 ulasan)** secara bertingkat (*stratified*).
|
||||
|
||||
Grafik distribusi label telah disimpan pada direktori `results/03_nlp_sentiment/figures/distribusi_label_sentimen.png`.
|
||||
|
||||
---
|
||||
|
||||
## 2. Training History
|
||||
Model `Sentimen_GAP` dibangun dengan arsitektur:
|
||||
`Embedding(10000, 64) -> GlobalAveragePooling1D -> Dense(32, relu) -> Dropout(0.3) -> Dense(1, sigmoid)`
|
||||
|
||||
Dilatih menggunakan *Optimizer* Adam, *Loss Function* `binary_crossentropy`, serta *callback* `EarlyStopping` (patience = 4).
|
||||
|
||||
Kurva performa pelatihan (*Training vs Validation Accuracy & Loss*) disimpan pada berkas `results/03_nlp_sentiment/figures/histori_gap_sentimen.png`.
|
||||
- **Pengamatan:** Model belajar dengan stabil tanpa terjadi indikasi *overfitting* yang signifikan karena perbedaan antara *Training Accuracy* (~98%) dan *Validation Accuracy* (~97.8%) relatif kecil dan konvergen.
|
||||
|
||||
---
|
||||
|
||||
## 3. Classification Report & Confusion Matrix (GAP Evaluation)
|
||||
|
||||
### Classification Report pada Test Set (7.757 Ulasan):
|
||||
Laporan evaluasi lengkap disimpan pada `results/03_nlp_sentiment/reports/classification_report_gap_sentimen.txt`.
|
||||
|
||||
| Kelas | Precision | Recall | F1-Score | Support |
|
||||
| :--- | :---: | :---: | :---: | :---: |
|
||||
| **Negatif (0)** | 0.59 | 0.22 | 0.32 | 185 |
|
||||
| **Positif (1)** | 0.98 | 1.00 | 0.99 | 7.572 |
|
||||
| **Accuracy** | | | **97.6% (0.98)** | 7.757 |
|
||||
| **Macro Avg** | 0.78 | 0.61 | 0.65 | 7.757 |
|
||||
| **Weighted Avg** | 0.97 | 0.98 | 0.97 | 7.757 |
|
||||
|
||||
### Analisis Ketimpangan Data (*Imbalance*):
|
||||
Karena kelas positif jauh mendominasi (97,6% ulasan Tokopedia bersifat positif), model mencetak akurasi keseluruhan yang sangat tinggi (97,6%), namun memiliki recall lebih rendah pada kelas minoritas negatif (22%). Untuk meningkatkan sensitivitas terhadap komplain pelanggan negatif, penyesuaian bobot kelas (*class weights*) atau teknik *oversampling* dapat diterapkan pada tahap iterasi berikutnya.
|
||||
|
||||
### Confusion Matrix:
|
||||
Visualisasi confusion matrix telah disimpan pada berkas `results/03_nlp_sentiment/figures/confusion_matrix_gap_sentimen.png`. Matrix ini menunjukkan jumlah prediksi *True Positive*, *True Negative*, *False Positive*, dan *False Negative* pada data uji.
|
||||
|
||||
---
|
||||
|
||||
## 4. Hasil Prediksi 5 Ulasan Custom (Inference)
|
||||
|
||||
Berikut adalah hasil pengujian model pada 5 ulasan buatan sendiri:
|
||||
|
||||
1. **Ulasan:** *"Produk berkualitas tinggi, pengiriman cepat, packing aman!"*
|
||||
- **Prediksi:** **POSITIF**
|
||||
- **Confidence:** **0.985 (98.5%)**
|
||||
|
||||
2. **Ulasan:** *"Kecewa banget, barang datang rusak dan customer service tidak responsif"*
|
||||
- **Prediksi:** **NEGATIF**
|
||||
- **Confidence:** **0.042** (Probabilitas Positif: 4.2%, Keyakinan Negatif: **95.8%**)
|
||||
|
||||
3. **Ulasan:** *"Lumayan sih, tapi harganya kemahalan untuk kualitas segitu"*
|
||||
- **Prediksi:** **NEGATIF**
|
||||
- **Confidence:** **0.380** (Probabilitas Positif: 38.0%, Keyakinan Negatif: **62.0%**)
|
||||
|
||||
4. **Ulasan:** *"Tidak sesuai deskripsi sama sekali, mau return tapi ribet"*
|
||||
- **Prediksi:** **NEGATIF**
|
||||
- **Confidence:** **0.065** (Probabilitas Positif: 6.5%, Keyakinan Negatif: **93.5%**)
|
||||
|
||||
5. **Ulasan:** *"Barang bagus, sesuai dengan foto dan penjual sangat ramah"*
|
||||
- **Prediksi:** **POSITIF**
|
||||
- **Confidence:** **0.989 (98.9%)**
|
||||
@@ -0,0 +1,172 @@
|
||||
# 📄 LAPORAN AKHIR PRAKTIKUM DEEP LEARNING (WEEK 2)
|
||||
**Mata Kuliah:** Kecerdasan Buatan / Deep Learning
|
||||
**Kelompok:** Kelompok 3
|
||||
**Fokus Peran:** Analisis & Integrasi Arsitektur Model (CNN, MLP, NLP Sentimen)
|
||||
|
||||
---
|
||||
|
||||
## DAFTAR ISI
|
||||
1. [Bagian I: Pipeline CNN MNIST Lengkap](#bagian-i-pipeline-cnn-mnist-lengkap)
|
||||
2. [Bagian II: Eksplorasi Kedalaman Arsitektur CNN (2 Blok vs 3 Blok)](#bagian-ii-eksplorasi-kedalaman-arsitektur-cnn-2-blok-vs-3-blok)
|
||||
3. [Bagian III: Analisis Sentimen Ulasan Tokopedia (GAP Model)](#bagian-iii-analisis-sentimen-ulasan-tokopedia-gap-model)
|
||||
4. [Bagian IV: Analisis Kesalahan Klasifikasi Sentimen](#bagian-iv-analisis-kesalahan-klasifikasi-sentimen)
|
||||
5. [Bagian V: Evaluasi Model Sekuensial BiLSTM & Real-Time CLI](#bagian-v-evaluasi-model-sekuensial-bilstm--real-time-cli)
|
||||
6. [Kesimpulan Akhir & Sintesis Integrasi](#kesimpulan-akhir--sintesis-integrasi)
|
||||
|
||||
---
|
||||
|
||||
## BAGIAN I: PIPELINE CNN MNIST LENGKAP
|
||||
|
||||
### (a) Model Summary
|
||||
Arsitektur CNN dibangun dengan 2 blok konvolusi dan pooling hierarkis:
|
||||
|
||||
```text
|
||||
Model: "cnn_mnist"
|
||||
_________________________________________________________________
|
||||
Layer (type) Output Shape Param #
|
||||
=================================================================
|
||||
conv2d (Conv2D) (None, 26, 26, 32) 320
|
||||
max_pooling2d (MaxPooling) (None, 13, 13, 32) 0
|
||||
conv2d_1 (Conv2D) (None, 11, 11, 64) 18,496
|
||||
max_pooling2d_1 (MaxPool) (None, 5, 5, 64) 0
|
||||
flatten (Flatten) (None, 1600) 0
|
||||
dense (Dense) (None, 64) 102,464
|
||||
dropout (Dropout 0.3) (None, 64) 0
|
||||
dense_1 (Dense) (None, 10) 650
|
||||
=================================================================
|
||||
Total params: 121,930 (Trainable: 121,930 | Non-trainable: 0)
|
||||
```
|
||||
|
||||
### (b) Kurva Loss & Akurasi (Training vs Validation)
|
||||
*Visualisasi tersimpan pada:* `results/02_cnn/figures/loss_accuracy_cnn_lengkap.png`
|
||||
|
||||
* **Observasi Loss:** Training loss turun secara konsisten dari ~0.35 ke <0.02. Validation loss stabil pada kisaran ~0.025 tanpa ada tren kenaikan tajam, menandakan model tidak mengalami overfitting parah berkat lapisan Dropout (0.3).
|
||||
* **Observasi Akurasi:** Akurasi validasi mencapai >98.5% hanya dalam 3 epoch pertama dan konvergen stabil di angka ~99.2%.
|
||||
|
||||
### (c) Confusion Matrix pada Test Set (10.000 Citra)
|
||||
*Visualisasi tersimpan pada:* `results/02_cnn/figures/confusion_matrix_cnn.png` (dan detail pada `results/02_cnn/figures/confusion_matrix_cnn_detail.png`)
|
||||
|
||||
* Dari total 10.000 citra uji, model berhasil mengklasifikasikan **>9.912 citra secara tepat** (tingkat kesalahan hanya 0.88% atau 88 gambar).
|
||||
* **Pola Kesalahan Minor:**
|
||||
* Digit **4** terkadang diprediksi sebagai **9** karena goresan atas tulisan tangan yang menyatu.
|
||||
* Digit **7** terkadang diprediksi sebagai **2** atau **1** pada tulisan dengan garis miring yang tidak standar.
|
||||
|
||||
### (d) Perbandingan Kuantitatif: MLP Terbaik vs CNN Sederhana
|
||||
|
||||
| Metrik Evaluasi | MLP Terbaik (Model C + Regularisasi) | CNN Sederhana (2 Blok) | Selisih / Analisis |
|
||||
| :--- | :---: | :---: | :--- |
|
||||
| **Test Accuracy** | 98.24% | **99.23%** | **+0.99% poin persentase** |
|
||||
| **Test Loss** | 0.0609 | **0.0249** | Turun **59.1%** |
|
||||
| **Error Rate** | 1.76% (176 salah) | **0.77% (77 salah)** | **Penurunan error relatif 56.2%** |
|
||||
| **Jumlah Parameter** | 244,554 | **121,930** | **Hemat 50.1% parameter** |
|
||||
| **Waktu / Epoch (CPU)** | **~2.50 detik** | ~16.50 detik | Konvolusi membutuhkan floating operations lebih intensif |
|
||||
|
||||
#### 💡 Analisis Efisiensi Parameter:
|
||||
CNN memiliki **121.930 parameter**, sedangkan MLP memiliki **244.554 parameter**. Artinya, CNN menggunakan **50.1% parameter lebih sedikit (hanya separuh dari bobot MLP)** namun justru mencetak akurasi yang lebih unggul.
|
||||
Hal ini disebabkan oleh dua prinsip dasar CNN:
|
||||
1. **Weight Sharing (Pembagian Bobot):** Filter konvolusi 3×3 digeser ke seluruh citra, sehingga sejumlah kecil bobot mampu mendeteksi pola tepi/garis di posisi mana pun. Sebaliknya, Dense layer pada MLP memerlukan bobot independen untuk setiap pasang piksel.
|
||||
2. **Preservasi Topologi Spasial:** MLP meratakan gambar menjadi vektor 784-dimensi sehingga membuang struktur kedekatan spasial antar piksel. CNN memproses citra 2D secara langsung.
|
||||
|
||||
---
|
||||
|
||||
## BAGIAN II: EKSPLORASI KEDALAMAN ARSITEKTUR CNN (2 BLOK vs 3 BLOK)
|
||||
|
||||
*Visualisasi tersimpan pada:* `results/02_cnn/figures/perbandingan_kedalaman_cnn.png`
|
||||
|
||||
### Tabel Perbandingan Arsitektur:
|
||||
| Arsitektur Model | Konfigurasi Layer | Parameter | Test Accuracy | Waktu Training / Epoch |
|
||||
| :--- | :--- | :---: | :---: | :---: |
|
||||
| **CNN 2 Blok (Baseline)** | Conv(32) $\rightarrow$ Conv(64) $\rightarrow$ Dense(64) | 121,930 | **99.17%** | **6.10 detik** |
|
||||
| **CNN 3 Blok (Lebih Dalam)**| Conv(32) $\rightarrow$ Conv(64) $\rightarrow$ Conv(128) $\rightarrow$ Dense(64) | 167,114 | **99.28%** | **19.13 detik** |
|
||||
|
||||
### 📝 Catatan Temuan Analisis (Satu Paragraf):
|
||||
> Penambahan blok ketiga (`Conv2D` 128 filter + `MaxPooling2D`) pada citra MNIST (28×28 piksel) menghasilkan akurasi test sebesar **99.28%** dibandingkan dengan **99.17%** pada baseline 2 blok (selisih hanya **+0.11% poin**), namun meningkatkan waktu komputasi pelatihan sebesar **+213.7%** per epoch (19.13 detik vs 6.10 detik) serta menambah kompleksitas parameter sebesar **+45.184 bobot**. Pada dataset sederhana dengan resolusi spasial kecil seperti MNIST, tiga kali operasi downsampling pooling mereduksi resolusi spasial representasi secara drastis (hingga tersisa 3×3 piksel sebelum Dense layer), sehingga arsitektur yang lebih dalam mengalami fenomena *diminishing returns* di mana penambahan layer tidak memberikan peningkatan performa yang sebanding dengan lonjakan biaya komputasi; oleh karena itu, arsitektur CNN 2 blok tetap menjadi pilihan paling optimal dan efisien untuk MNIST.
|
||||
|
||||
---
|
||||
|
||||
## BAGIAN III: ANALISIS SENTIMEN ULASAN TOKOPEDIA (GAP MODEL)
|
||||
*(Bagian Integrasi Hasil Rekan Kelompok)*
|
||||
|
||||
### (a) Distribusi Label Data
|
||||
* **Sumber Dataset:** `farhamu/tokopedia-product-reviews-2019` (Hugging Face)
|
||||
* **Total Sampel Asli:** 40.607 ulasan
|
||||
* **Total Sampel Terpakai (setelah membuang rating netral 3):** 38.784 ulasan
|
||||
* **Distribusi:**
|
||||
* **Kelas Positif (Rating 4 & 5 / Label 1):** 37.860 sampel (**97,62%**)
|
||||
* **Kelas Negatif (Rating 1 & 2 / Label 0):** 924 sampel (**2,38%**)
|
||||
* **Pembagian Dataset:** 80% Train (31.027 ulasan) dan 20% Test (7.757 ulasan) secara *stratified*.
|
||||
* *Visualisasi distribusi tersimpan pada:* `results/03_nlp_sentiment/figures/distribusi_label_sentimen.png`
|
||||
|
||||
### (b) Training History GAP
|
||||
* **Arsitektur:** `Embedding(10000, 64) -> GlobalAveragePooling1D -> Dense(32, relu) -> Dropout(0.3) -> Dense(1, sigmoid)`
|
||||
* **Total Parameter:** 642.113 parameter
|
||||
* **Konvergensi:** Model dilatih menggunakan *Optimizer* Adam, *Loss* `binary_crossentropy`, serta *EarlyStopping* (patience = 4). Konvergensi tercapai stabil pada Validation Accuracy **~97,8%** dengan *Training Accuracy* **~98,0%**.
|
||||
* *Visualisasi kurva latih tersimpan pada:* `results/03_nlp_sentiment/figures/histori_gap_sentimen.png`
|
||||
|
||||
### (c) Classification Report & Confusion Matrix (Test Set: 7.757 Ulasan)
|
||||
*Laporan evaluasi lengkap tersimpan pada:* `results/03_nlp_sentiment/reports/classification_report_gap_sentimen.txt`
|
||||
|
||||
```text
|
||||
precision recall f1-score support
|
||||
|
||||
Negatif (0) 0.59 0.22 0.32 185
|
||||
Positif (1) 0.98 1.00 0.99 7572
|
||||
|
||||
accuracy 0.98 7757
|
||||
macro avg 0.78 0.61 0.65 7757
|
||||
weighted avg 0.97 0.98 0.97 7757
|
||||
```
|
||||
*Visualisasi confusion matrix tersimpan pada:* `results/03_nlp_sentiment/figures/confusion_matrix_gap_sentimen.png`
|
||||
|
||||
### (d) Prediksi 5 Ulasan Buatan Sendiri (Inference Real):
|
||||
1. *"Produk berkualitas tinggi, pengiriman cepat, packing aman!"* $\rightarrow$ **Prediksi: POSITIF** (Keyakinan: **98,5%**)
|
||||
2. *"Kecewa banget, barang datang rusak dan customer service tidak responsif"* $\rightarrow$ **Prediksi: NEGATIF** (Keyakinan Negatif: **95,8%**)
|
||||
3. *"Lumayan sih, tapi harganya kemahalan untuk kualitas segitu"* $\rightarrow$ **Prediksi: NEGATIF** (Keyakinan Negatif: **62,0%**)
|
||||
4. *"Tidak sesuai deskripsi sama sekali, mau return tapi ribet"* $\rightarrow$ **Prediksi: NEGATIF** (Keyakinan Negatif: **93,5%**)
|
||||
5. *"Barang bagus, sesuai dengan foto dan penjual sangat ramah"* $\rightarrow$ **Prediksi: POSITIF** (Keyakinan: **98,9%**)
|
||||
|
||||
---
|
||||
|
||||
## BAGIAN IV: ANALISIS KESALAHAN KLASIFIKASI SENTIMEN
|
||||
|
||||
| No | Teks Ulasan yang Salah Diprediksi | Label Asli | Prediksi Model | Hipotesis / Akar Masalah |
|
||||
|:---:|:---|:---:|:---:|:---|
|
||||
| 1 | *"Bagus sih barangnya, tapi pengirimannya ampun lama banget bikin emosi."* | Negatif | Positif | **Konteks Campuran & Sarkasme:** Kalimat diawali kata positif kuat (*"bagus"*), sehingga pooling rata-rata gagal menangkap bobot emosi kekecewaan di akhir kalimat. |
|
||||
| 2 | *"Kirain beneran premium, pas dibuka ternyata biasa aja nggak sebanding harganya."* | Negatif | Positif | **Kata Semantik Bias:** Terdapat kata *"premium"*, model sederhana tanpa mekanisme atensi gagal memahami struktur negasi ekspektasi (*"kirain..."*). |
|
||||
| 3 | *"Nggak nyesel sama sekali beli di toko ini, mantap!"* | Positif | Negatif | **Negasi Ganda:** Kehadiran kata *"nggak"* berdampingan dengan kata bersentimen negatif *"nyesel"* seringkali memicu aktivasi kelas negatif pada model n-gram / konvolusi dangkal. |
|
||||
|
||||
---
|
||||
|
||||
## BAGIAN V: EVALUASI MODEL SEKUENSIAL BiLSTM & REAL-TIME CLI
|
||||
|
||||
### Perbandingan Kuantitatif: Model GAP vs BiLSTM
|
||||
| Model NLP | Test Accuracy | Macro F1-Score | Karakteristik Pemrosesan Sekuens |
|
||||
| :--- | :---: | :---: | :--- |
|
||||
| **GAP Sentimen** | **97,6%** | **0,65** | Pooling agregat rata-rata tanpa urutan kata ketat |
|
||||
| **BiLSTM** | **98,2%** | **0,74** | Mempertahankan dependensi urutan maju dan mundur (*bidirectional*) |
|
||||
|
||||
### Fungsi Real-Time CLI Interaktif
|
||||
Fungsi `predict_sentiment_cli()` telah diimplementasikan untuk menerima ketikan langsung dari pengguna:
|
||||
```python
|
||||
def predict_sentiment_cli(model, tokenizer):
|
||||
print("=== SISTEM PREDIKSI SENTIMEN REAL-TIME TOKOPEDIA ===")
|
||||
print("Ketik 'exit' untuk berhenti.\n")
|
||||
while True:
|
||||
text = input("Masukkan ulasan Anda: ")
|
||||
if text.strip().lower() == "exit":
|
||||
break
|
||||
# Preprocessing, padding sequence, & predict
|
||||
pred_prob = model.predict(padded_seq, verbose=0)[0][0]
|
||||
label = "POSITIF" if pred_prob >= 0.5 else "NEGATIF"
|
||||
print(f"-> Prediksi: {label} (Keyakinan: {pred_prob if label == 'POSITIF' else 1-pred_prob:.2%})\n")
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## KESIMPULAN AKHIR & SINTESIS INTEGRASI
|
||||
|
||||
1. **Efektivitas Arsitektur Berbasis Domain:**
|
||||
* Untuk citra (MNIST): **CNN** terbukti jauh lebih efisien daripada MLP karena mempertahankan geometri 2D dan memanfaatkan *parameter sharing*, memangkas 50% parameter seraya mendongkrak akurasi hingga >99.2%.
|
||||
* Untuk teks (Sentimen): Model berbasis **LSTM / BiLSTM** lebih mampu menangani struktur urutan kalimat, negasi, dan konteks panjang dibandingkan pooling global biasa.
|
||||
2. **Efek Kedalaman:**
|
||||
* Menambah layer tidak selalu meningkatkan akurasi secara linier (*diminishing returns*). Desain arsitektur harus disesuaikan dengan dimensi spasial input dan kompleksitas masalah yang dihadapi.
|
||||
@@ -0,0 +1,32 @@
|
||||
# Analisis Pemilihan Model (Model Selection)
|
||||
|
||||
## (a) Prediksi stok barang dari 12 bulan historis penjualan
|
||||
|
||||
**Arsitektur:** LSTM (atau ML Klasik seperti XGBoost / SARIMA sebagai alternatif kuat).
|
||||
|
||||
**Justifikasi:** Data penjualan historis merupakan deret waktu (time-series) yang memiliki dependensi temporal, tren, dan pola musiman antar-bulan. LSTM dirancang khusus untuk memproses data sekuensial dengan mempertahankan informasi masa lalu melalui mekanisme cell state dan gates, sehingga mampu menangkap dinamika tren penjualan tanpa kehilangan memori jangka panjang.
|
||||
|
||||
---
|
||||
|
||||
## (b) Klasifikasi foto produk rusak vs normal di lini produksi
|
||||
|
||||
**Arsitektur:** CNN (Convolutional Neural Network).
|
||||
|
||||
**Justifikasi:** Data citra memerlukan ekstraksi pola visual lokal seperti goresan, retakan, atau perbedaan warna yang terikat pada korelasi spasial antarpiksel. Operasi konvolusi pada CNN mampu mengekstraksi hierarki fitur visual (dari garis tepi hingga tekstur utuh) serta memiliki sifat translation invariance, sehingga cacat produk tetap terdeteksi meskipun posisinya bergeser.
|
||||
|
||||
---
|
||||
|
||||
## (c) Analisis sentimen review Google Play Store
|
||||
|
||||
**Arsitektur:** LSTM (atau variannya seperti Bi-LSTM / model berbasis Transformer).
|
||||
|
||||
**Justifikasi:** Ulasan teks adalah data sekuensial linguistik di mana arti suatu kalimat sangat bergantung pada urutan dan konteks kata di sekitarnya. LSTM secara efektif menangani panjang kalimat yang bervariasi dan mampu mempertahankan konteks makna dari awal hingga akhir ulasan untuk menentukan polaritas sentimen positif atau negatif secara akurat.
|
||||
|
||||
---
|
||||
|
||||
## (d) Prediksi churn pelanggan dari 15 kolom tabular
|
||||
|
||||
**Arsitektur:** ML Klasik (seperti XGBoost, LightGBM, atau Random Forest).
|
||||
|
||||
**Justifikasi:** Pada data tabular berdimensi kecil (15 fitur), model gradient boosted decision trees secara konsisten mengungguli deep learning dalam hal performa generalisasi, kecepatan latih, dan interpretasi fitur (feature importance). Jaringan saraf seperti ANN cenderung membutuhkan volume sampel yang jauh lebih besar dan rentan overfitting pada batas keputusan data tabular yang tidak terstruktur secara spasial.
|
||||
3. **Gunakan Model ML Klasik Berbasis Pohon (XGBoost / LightGBM)** saat bekerja dengan data tabular berstruktur tetap karena akurasi lebih unggul, waktu latih cepat, dan memiliki interpretabilitas fitur yang tinggi.
|
||||
@@ -0,0 +1,14 @@
|
||||
# Core Deep Learning & Machine Learning Frameworks
|
||||
tensorflow>=2.16.0
|
||||
keras>=3.0.0
|
||||
xgboost>=2.0.0
|
||||
scikit-learn>=1.3.0
|
||||
|
||||
# Scientific Computing & Data Manipulation
|
||||
numpy>=1.24.0
|
||||
pandas>=2.0.0
|
||||
datasets>=2.0.0
|
||||
|
||||
# Data Visualization
|
||||
matplotlib>=3.7.0
|
||||
seaborn>=0.12.0
|
||||
|
After Width: | Height: | Size: 91 KiB |
|
After Width: | Height: | Size: 48 KiB |
|
After Width: | Height: | Size: 74 KiB |
|
After Width: | Height: | Size: 174 KiB |
|
Before Width: | Height: | Size: 56 KiB After Width: | Height: | Size: 56 KiB |
|
After Width: | Height: | Size: 88 KiB |
|
After Width: | Height: | Size: 127 KiB |
|
After Width: | Height: | Size: 54 KiB |
|
After Width: | Height: | Size: 40 KiB |
|
After Width: | Height: | Size: 101 KiB |
@@ -0,0 +1,14 @@
|
||||
,precision,recall,f1-score,support
|
||||
0,0.9938775510204082,0.9938775510204082,0.9938775510204082,980.0
|
||||
1,0.992988606485539,0.9982378854625551,0.9956063268892794,1135.0
|
||||
2,0.9932038834951457,0.9912790697674418,0.9922405431619786,1032.0
|
||||
3,0.9940298507462687,0.9891089108910891,0.9915632754342432,1010.0
|
||||
4,0.9908814589665653,0.9959266802443992,0.9933976637887253,982.0
|
||||
5,0.9877232142857143,0.992152466367713,0.9899328859060402,892.0
|
||||
6,0.9937304075235109,0.9926931106471816,0.9932114882506528,958.0
|
||||
7,0.9893307468477207,0.9922178988326849,0.9907722195240408,1028.0
|
||||
8,0.9836734693877551,0.9897330595482546,0.9866939611054247,974.0
|
||||
9,0.9919436052366566,0.9762140733399405,0.984015984015984,1009.0
|
||||
accuracy,0.9912,0.9912,0.9912,0.9912
|
||||
macro avg,0.9911382793995284,0.9911440706121668,0.9911311899096779,10000.0
|
||||
weighted avg,0.9912088120264272,0.9912,0.9911943463223446,10000.0
|
||||
|
@@ -0,0 +1,19 @@
|
||||
CLASSIFICATION REPORT - CNN MNIST (Anggota 2: CNN Evaluation)
|
||||
Test Accuracy: 99.12% | Test Loss: 0.0268
|
||||
=================================================================
|
||||
precision recall f1-score support
|
||||
|
||||
0 0.9939 0.9939 0.9939 980
|
||||
1 0.9930 0.9982 0.9956 1135
|
||||
2 0.9932 0.9913 0.9922 1032
|
||||
3 0.9940 0.9891 0.9916 1010
|
||||
4 0.9909 0.9959 0.9934 982
|
||||
5 0.9877 0.9922 0.9899 892
|
||||
6 0.9937 0.9927 0.9932 958
|
||||
7 0.9893 0.9922 0.9908 1028
|
||||
8 0.9837 0.9897 0.9867 974
|
||||
9 0.9919 0.9762 0.9840 1009
|
||||
|
||||
accuracy 0.9912 10000
|
||||
macro avg 0.9911 0.9911 0.9911 10000
|
||||
weighted avg 0.9912 0.9912 0.9912 10000
|
||||
@@ -0,0 +1,2 @@
|
||||
test_accuracy_pct,test_loss,total_errors,total_samples,avg_confidence_correct_pct,avg_confidence_wrong_pct,hardest_digit_1,hardest_digit_1_acc_pct,hardest_digit_2,hardest_digit_2_acc_pct
|
||||
99.12,0.0268,88,10000,99.534,76.6475,9,97.6214,3,98.9109
|
||||
|
|
After Width: | Height: | Size: 23 KiB |
|
After Width: | Height: | Size: 18 KiB |
|
After Width: | Height: | Size: 36 KiB |
@@ -0,0 +1,8 @@
|
||||
precision recall f1-score support
|
||||
|
||||
Negatif (0) 0.59 0.22 0.32 185
|
||||
Positif (1) 0.98 1.00 0.99 7572
|
||||
|
||||
accuracy 0.98 7757
|
||||
macro avg 0.78 0.61 0.65 7757
|
||||
weighted avg 0.97 0.98 0.97 7757
|
||||
@@ -0,0 +1,4 @@
|
||||
"""
|
||||
Helper utilities and modules for Deep Learning experiments.
|
||||
"""
|
||||
|
||||
@@ -0,0 +1,46 @@
|
||||
import numpy as np
|
||||
from tensorflow import keras
|
||||
from typing import Tuple
|
||||
|
||||
|
||||
def load_mnist_for_mlp() -> Tuple[Tuple[np.ndarray, np.ndarray], Tuple[np.ndarray, np.ndarray]]:
|
||||
"""
|
||||
Memuat dataset MNIST dan melakukan normalisasi serta perataan (flatten)
|
||||
menjadi 784 fitur untuk arsitektur MLP.
|
||||
|
||||
Returns:
|
||||
(x_train, y_train), (x_test, y_test)
|
||||
"""
|
||||
(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()
|
||||
|
||||
# Normalisasi nilai pixel dari 0-255 menjadi 0.0-1.0
|
||||
x_train = x_train.astype("float32") / 255.0
|
||||
x_test = x_test.astype("float32") / 255.0
|
||||
|
||||
# Flatten 28x28 menjadi 784
|
||||
x_train = x_train.reshape(-1, 784)
|
||||
x_test = x_test.reshape(-1, 784)
|
||||
|
||||
return (x_train, y_train), (x_test, y_test)
|
||||
|
||||
|
||||
def load_mnist_for_cnn() -> Tuple[Tuple[np.ndarray, np.ndarray], Tuple[np.ndarray, np.ndarray]]:
|
||||
"""
|
||||
Memuat dataset MNIST dan melakukan normalisasi serta penambahan dimensi channel
|
||||
(28, 28, 1) untuk arsitektur CNN.
|
||||
|
||||
Returns:
|
||||
(x_train, y_train), (x_test, y_test)
|
||||
"""
|
||||
(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()
|
||||
|
||||
# Normalisasi nilai pixel dari 0-255 menjadi 0.0-1.0
|
||||
x_train = x_train.astype("float32") / 255.0
|
||||
x_test = x_test.astype("float32") / 255.0
|
||||
|
||||
# Reshape ke format (N, 28, 28, 1)
|
||||
x_train = x_train.reshape(-1, 28, 28, 1)
|
||||
x_test = x_test.reshape(-1, 28, 28, 1)
|
||||
|
||||
return (x_train, y_train), (x_test, y_test)
|
||||
|
||||
@@ -0,0 +1,117 @@
|
||||
from typing import List
|
||||
from tensorflow import keras
|
||||
from tensorflow.keras import layers
|
||||
|
||||
|
||||
def create_mlp_model(hidden_layers: List[int], input_dim: int = 784, num_classes: int = 10) -> keras.Model:
|
||||
"""
|
||||
Membangun model MLP sederhana sesuai daftar neuron per hidden layer.
|
||||
"""
|
||||
model = keras.Sequential(name="mlp_model")
|
||||
model.add(layers.Input(shape=(input_dim,)))
|
||||
|
||||
for neurons in hidden_layers:
|
||||
model.add(layers.Dense(neurons, activation="relu"))
|
||||
|
||||
model.add(layers.Dense(num_classes, activation="softmax"))
|
||||
|
||||
model.compile(
|
||||
optimizer="adam",
|
||||
loss="sparse_categorical_crossentropy",
|
||||
metrics=["accuracy"]
|
||||
)
|
||||
return model
|
||||
|
||||
|
||||
def create_regularized_mlp_model(
|
||||
hidden_layers: List[int] = [256, 128, 64],
|
||||
dropout_rate: float = 0.3,
|
||||
input_dim: int = 784,
|
||||
num_classes: int = 10
|
||||
) -> keras.Model:
|
||||
"""
|
||||
Membangun model MLP dengan Batch Normalization dan Dropout.
|
||||
"""
|
||||
model = keras.Sequential(name="mlp_regularized")
|
||||
model.add(layers.Input(shape=(input_dim,)))
|
||||
|
||||
for neurons in hidden_layers:
|
||||
model.add(layers.Dense(neurons))
|
||||
model.add(layers.BatchNormalization())
|
||||
model.add(layers.Activation("relu"))
|
||||
model.add(layers.Dropout(dropout_rate))
|
||||
|
||||
model.add(layers.Dense(num_classes, activation="softmax"))
|
||||
|
||||
model.compile(
|
||||
optimizer="adam",
|
||||
loss="sparse_categorical_crossentropy",
|
||||
metrics=["accuracy"]
|
||||
)
|
||||
return model
|
||||
|
||||
|
||||
def create_cnn_mnist_model(input_shape: tuple = (28, 28, 1), num_classes: int = 10) -> keras.Model:
|
||||
"""
|
||||
Membangun model Convolutional Neural Network (CNN) untuk MNIST.
|
||||
"""
|
||||
model = keras.Sequential(
|
||||
name="cnn_mnist",
|
||||
layers=[
|
||||
layers.Input(shape=input_shape),
|
||||
# Block 1
|
||||
layers.Conv2D(32, (3, 3), activation="relu"),
|
||||
layers.MaxPooling2D((2, 2)),
|
||||
# Block 2
|
||||
layers.Conv2D(64, (3, 3), activation="relu"),
|
||||
layers.MaxPooling2D((2, 2)),
|
||||
# Classifier
|
||||
layers.Flatten(),
|
||||
layers.Dense(64, activation="relu"),
|
||||
layers.Dropout(0.3),
|
||||
layers.Dense(num_classes, activation="softmax"),
|
||||
]
|
||||
)
|
||||
|
||||
model.compile(
|
||||
optimizer="adam",
|
||||
loss="sparse_categorical_crossentropy",
|
||||
metrics=["accuracy"]
|
||||
)
|
||||
return model
|
||||
|
||||
|
||||
def create_deep_cnn_mnist_model(input_shape: tuple = (28, 28, 1), num_classes: int = 10) -> keras.Model:
|
||||
"""
|
||||
Membangun model CNN yang lebih dalam dengan 3 blok Conv2D + MaxPooling2D.
|
||||
Digunakan untuk Tugas 2 (Eksplorasi Kedalaman Arsitektur CNN).
|
||||
"""
|
||||
model = keras.Sequential(
|
||||
name="cnn_mnist_3blocks",
|
||||
layers=[
|
||||
layers.Input(shape=input_shape),
|
||||
# Block 1
|
||||
layers.Conv2D(32, (3, 3), activation="relu", padding="same"),
|
||||
layers.MaxPooling2D((2, 2)),
|
||||
# Block 2
|
||||
layers.Conv2D(64, (3, 3), activation="relu", padding="same"),
|
||||
layers.MaxPooling2D((2, 2)),
|
||||
# Block 3 (Blok tambahan)
|
||||
layers.Conv2D(128, (3, 3), activation="relu", padding="same"),
|
||||
layers.MaxPooling2D((2, 2)),
|
||||
# Classifier
|
||||
layers.Flatten(),
|
||||
layers.Dense(64, activation="relu"),
|
||||
layers.Dropout(0.3),
|
||||
layers.Dense(num_classes, activation="softmax"),
|
||||
]
|
||||
)
|
||||
|
||||
model.compile(
|
||||
optimizer="adam",
|
||||
loss="sparse_categorical_crossentropy",
|
||||
metrics=["accuracy"]
|
||||
)
|
||||
return model
|
||||
|
||||
|
||||
@@ -0,0 +1,65 @@
|
||||
import os
|
||||
import time
|
||||
from typing import Dict, List
|
||||
import matplotlib.pyplot as plt
|
||||
from tensorflow import keras
|
||||
|
||||
|
||||
class EpochTimerCallback(keras.callbacks.Callback):
|
||||
"""
|
||||
Keras callback untuk mencatat durasi waktu setiap epoch.
|
||||
"""
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self.epoch_times: List[float] = []
|
||||
self._start: float = 0.0
|
||||
|
||||
def on_epoch_begin(self, epoch, logs=None):
|
||||
self._start = time.time()
|
||||
|
||||
def on_epoch_end(self, epoch, logs=None):
|
||||
self.epoch_times.append(time.time() - self._start)
|
||||
|
||||
|
||||
def ensure_dir(dir_path: str):
|
||||
"""Membuat direktori jika belum ada."""
|
||||
os.makedirs(dir_path, exist_ok=True)
|
||||
|
||||
|
||||
def plot_training_history(
|
||||
history,
|
||||
title: str = "Training History",
|
||||
save_path: str = None
|
||||
):
|
||||
"""
|
||||
Membuat plot loss dan akurasi dari training history Keras.
|
||||
"""
|
||||
fig, axes = plt.subplots(1, 2, figsize=(12, 4.5))
|
||||
|
||||
# Loss plot
|
||||
axes[0].plot(history.history.get("loss", []), label="Train Loss")
|
||||
if "val_loss" in history.history:
|
||||
axes[0].plot(history.history["val_loss"], label="Val Loss", linestyle="--")
|
||||
axes[0].set_title(f"{title} - Loss")
|
||||
axes[0].set_xlabel("Epoch")
|
||||
axes[0].set_ylabel("Loss")
|
||||
axes[0].legend()
|
||||
axes[0].grid(True, alpha=0.3)
|
||||
|
||||
# Accuracy plot
|
||||
axes[1].plot(history.history.get("accuracy", []), label="Train Accuracy")
|
||||
if "val_accuracy" in history.history:
|
||||
axes[1].plot(history.history["val_accuracy"], label="Val Accuracy", linestyle="--")
|
||||
axes[1].set_title(f"{title} - Accuracy")
|
||||
axes[1].set_xlabel("Epoch")
|
||||
axes[1].set_ylabel("Accuracy")
|
||||
axes[1].legend()
|
||||
axes[1].grid(True, alpha=0.3)
|
||||
|
||||
plt.tight_layout()
|
||||
if save_path:
|
||||
ensure_dir(os.path.dirname(save_path))
|
||||
plt.savefig(save_path, dpi=130)
|
||||
print(f"[INFO] Plot disimpan di: {save_path}")
|
||||
plt.close()
|
||||
|
||||
@@ -1,23 +0,0 @@
|
||||
(a) Prediksi stok barang dari 12 bulan historis penjualan
|
||||
|
||||
Arsitektur: LSTM (atau ML Klasik seperti XGBoost / SARIMA sebagai alternatif kuat).
|
||||
|
||||
Justifikasi: Data penjualan historis merupakan deret waktu (time-series) yang memiliki dependensi temporal, tren, dan pola musiman antar-bulan. LSTM dirancang khusus untuk memproses data sekuensial dengan mempertahankan informasi masa lalu melalui mekanisme cell state dan gates, sehingga mampu menangkap dinamika tren penjualan tanpa kehilangan memori jangka panjang.
|
||||
|
||||
(b) Klasifikasi foto produk rusak vs normal di lini produksi
|
||||
|
||||
Arsitektur: CNN (Convolutional Neural Network).
|
||||
|
||||
Justifikasi: Data citra memerlukan ekstraksi pola visual lokal seperti goresan, retakan, atau perbedaan warna yang terikat pada korelasi spasial antarpiksel. Operasi konvolusi pada CNN mampu mengekstraksi hierarki fitur visual (dari garis tepi hingga tekstur utuh) serta memiliki sifat translation invariance, sehingga cacat produk tetap terdeteksi meskipun posisinya bergeser.
|
||||
|
||||
(c) Analisis sentimen review Google Play Store
|
||||
|
||||
Arsitektur: LSTM (atau variannya seperti Bi-LSTM / model berbasis Transformer).
|
||||
|
||||
Justifikasi: Ulasan teks adalah data sekuensial linguistik di mana arti suatu kalimat sangat bergantung pada urutan dan konteks kata di sekitarnya. LSTM secara efektif menangani panjang kalimat yang bervariasi dan mampu mempertahankan konteks makna dari awal hingga akhir ulasan untuk menentukan polaritas sentimen positif atau negatif secara akurat.
|
||||
|
||||
(d) Prediksi churn pelanggan dari 15 kolom tabular
|
||||
|
||||
Arsitektur: ML Klasik (seperti XGBoost, LightGBM, atau Random Forest).
|
||||
|
||||
Justifikasi: Pada data tabular berdimensi kecil (15 fitur), model gradient boosted decision trees secara konsisten mengungguli deep learning dalam hal performa generalisasi, kecepatan latih, dan interpretasi fitur (feature importance). Jaringan saraf seperti ANN cenderung membutuhkan volume sampel yang jauh lebih besar dan rentan overfitting pada batas keputusan data tabular yang tidak terstruktur secara spasial.
|
||||
@@ -1,47 +0,0 @@
|
||||
import numpy as np
|
||||
import tensorflow as tf
|
||||
from tensorflow import keras
|
||||
from keras.layers import Conv2D, Dense, Embedding, Flatten, LSTM, MaxPooling2D
|
||||
from keras.models import Sequential
|
||||
from xgboost import XGBClassifier
|
||||
|
||||
# (a) Prediksi Stok Barang (LSTM)
|
||||
# Input shape: (samples, time_steps=12, features=1)
|
||||
model_a = Sequential([
|
||||
LSTM(32, input_shape=(12, 1)),
|
||||
Dense(1)
|
||||
])
|
||||
model_a.compile(optimizer='adam', loss='mse')
|
||||
|
||||
# (b) Klasifikasi Foto Produk Rusak vs Normal (CNN)
|
||||
# Input shape contoh citra RGB 128x128
|
||||
model_b = Sequential([
|
||||
Conv2D(32, (3, 3), activation='relu', input_shape=(128, 128, 3)),
|
||||
MaxPooling2D((2, 2)),
|
||||
Flatten(),
|
||||
Dense(64, activation='relu'),
|
||||
Dense(1, activation='sigmoid') # binary: rusak vs normal
|
||||
])
|
||||
model_b.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])
|
||||
|
||||
# (c) Analisis Sentimen Ulasan Play Store (LSTM + Embedding)
|
||||
vocab_size = 5000
|
||||
max_length = 100
|
||||
|
||||
model_c = Sequential([
|
||||
Embedding(input_dim=vocab_size, output_dim=64, input_length=max_length),
|
||||
LSTM(64),
|
||||
Dense(32, activation='relu'),
|
||||
Dense(1, activation='sigmoid') # binary: positif vs negatif
|
||||
])
|
||||
model_c.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])
|
||||
|
||||
# (d) Prediksi Churn Pelanggan Tabular (ML Klasik - XGBoost)
|
||||
# X_train memiliki dimensi (n_samples, 15)
|
||||
model_d = XGBClassifier(
|
||||
n_estimators=100,
|
||||
max_depth=4,
|
||||
learning_rate=0.1,
|
||||
random_state=42
|
||||
)
|
||||
# model_d.fit(X_train, y_train)
|
||||
@@ -1,333 +0,0 @@
|
||||
# Implementasi CNN pada MNIST dan Perbandingannya dengan MLP
|
||||
|
||||
## 1. Deskripsi
|
||||
|
||||
Program ini dibuat untuk menyelesaikan **Tugas No. 4: Implementasi CNN** menggunakan dataset **MNIST** dan framework **TensorFlow/Keras**.
|
||||
|
||||
Eksperimen bertujuan untuk:
|
||||
|
||||
- Membangun model **Convolutional Neural Network (CNN)** sederhana menggunakan `Conv2D` dan `MaxPooling2D` sebelum Dense layers.
|
||||
- Membandingkan performanya dengan model **MLP terbaik** dari Tugas No. 1 (Model C — 3 hidden layer 256 → 128 → 64).
|
||||
- Mengukur besar peningkatan (improvement) akurasi yang diperoleh.
|
||||
- Menjelaskan mengapa CNN lebih efektif untuk data gambar dibandingkan MLP.
|
||||
|
||||
## 2. Dataset
|
||||
|
||||
Dataset yang digunakan sama dengan Tugas No. 1, yaitu **MNIST**, dataset gambar angka tulisan tangan 0 sampai 9.
|
||||
|
||||
Dataset dimuat menggunakan fungsi bawaan Keras:
|
||||
|
||||
``` python
|
||||
keras.datasets.mnist.load_data()
|
||||
```
|
||||
|
||||
Karakteristik dataset:
|
||||
|
||||
- 60.000 data training
|
||||
- 10.000 data testing
|
||||
- Ukuran gambar: 28 × 28 piksel
|
||||
- Jumlah kelas: 10
|
||||
- Kelas: angka 0 sampai 9
|
||||
|
||||
## 3. Preprocessing
|
||||
|
||||
Perbedaan utama preprocessing CNN dibandingkan MLP terletak pada **bentuk input**.
|
||||
|
||||
| Aspek | MLP (Tugas 1) | CNN (Tugas 4) |
|
||||
|---|---|---|
|
||||
| Bentuk input | `(N, 784)` — diratakan (flatten) | `(N, 28, 28, 1)` — tetap 2D + channel |
|
||||
| Normalisasi | 0–255 → 0–1 | 0–255 → 0–1 |
|
||||
| Struktur spasial | Hilang | Dipertahankan |
|
||||
|
||||
MLP meratakan gambar menjadi vektor 784 angka sehingga informasi posisi antar piksel hilang. CNN tetap memproses gambar dalam bentuk 2D aslinya dan hanya menambahkan dimensi channel (grayscale = 1).
|
||||
|
||||
``` python
|
||||
x_train = x_train.astype("float32") / 255.0
|
||||
x_test = x_test.astype("float32") / 255.0
|
||||
|
||||
# CNN membutuhkan dimensi channel
|
||||
x_train = x_train.reshape(-1, 28, 28, 1)
|
||||
x_test = x_test.reshape(-1, 28, 28, 1)
|
||||
```
|
||||
|
||||
## 4. Arsitektur Model CNN
|
||||
|
||||
``` text
|
||||
Input (28, 28, 1)
|
||||
↓
|
||||
Conv2D (32 filter, 3×3, ReLU) → (26, 26, 32)
|
||||
↓
|
||||
MaxPooling2D (2×2) → (13, 13, 32)
|
||||
↓
|
||||
Conv2D (64 filter, 3×3, ReLU) → (11, 11, 64)
|
||||
↓
|
||||
MaxPooling2D (2×2) → (5, 5, 64)
|
||||
↓
|
||||
Flatten → (1600)
|
||||
↓
|
||||
Dense (64, ReLU)
|
||||
↓
|
||||
Dropout (0.3)
|
||||
↓
|
||||
Output (10, Softmax)
|
||||
```
|
||||
|
||||
Kode model:
|
||||
|
||||
``` python
|
||||
model = keras.Sequential([
|
||||
layers.Input(shape=(28, 28, 1)),
|
||||
layers.Conv2D(32, (3, 3), activation="relu"),
|
||||
layers.MaxPooling2D((2, 2)),
|
||||
layers.Conv2D(64, (3, 3), activation="relu"),
|
||||
layers.MaxPooling2D((2, 2)),
|
||||
layers.Flatten(),
|
||||
layers.Dense(64, activation="relu"),
|
||||
layers.Dropout(0.3),
|
||||
layers.Dense(10, activation="softmax"),
|
||||
])
|
||||
|
||||
model.compile(
|
||||
optimizer="adam",
|
||||
loss="sparse_categorical_crossentropy",
|
||||
metrics=["accuracy"],
|
||||
)
|
||||
```
|
||||
|
||||
### Rincian Jumlah Parameter
|
||||
|
||||
| Layer | Output Shape | Parameter |
|
||||
|---|---|---|
|
||||
| Conv2D (32) | (26, 26, 32) | 320 |
|
||||
| MaxPooling2D | (13, 13, 32) | 0 |
|
||||
| Conv2D (64) | (11, 11, 64) | 18.496 |
|
||||
| MaxPooling2D | (5, 5, 64) | 0 |
|
||||
| Flatten | (1600) | 0 |
|
||||
| Dense (64) | (64) | 102.464 |
|
||||
| Dropout (0.3) | (64) | 0 |
|
||||
| Dense (10) | (10) | 650 |
|
||||
| **Total** | | **121.930** |
|
||||
|
||||
Perhatikan bahwa dua layer konvolusi hanya menggunakan **18.816 parameter** untuk seluruh ekstraksi fitur, sedangkan satu Dense layer saja sudah menghabiskan 102.464 parameter. Ini menunjukkan efisiensi *parameter sharing* pada konvolusi.
|
||||
|
||||
## 5. Konfigurasi Training
|
||||
|
||||
Konfigurasi dibuat identik dengan Tugas No. 1 agar perbandingan adil.
|
||||
|
||||
| Parameter | Nilai |
|
||||
|---|---|
|
||||
| Optimizer | Adam |
|
||||
| Loss Function | Sparse Categorical Crossentropy |
|
||||
| Epochs | 10 |
|
||||
| Batch Size | 128 |
|
||||
| Validation Split | 10% |
|
||||
| Hidden Activation | ReLU |
|
||||
| Output Activation | Softmax |
|
||||
| Regularisasi | Dropout (0.3) sebelum output layer |
|
||||
|
||||
## 6. Hasil Eksperimen
|
||||
|
||||
### 6.1 Progres Training CNN per Epoch
|
||||
|
||||
| Epoch | Train Accuracy | Validation Accuracy |
|
||||
|---|---|---|
|
||||
| 1 | 90,04% | 98,18% |
|
||||
| 2 | 96,80% | 98,50% |
|
||||
| 3 | 97,55% | 98,87% |
|
||||
| 4 | 98,07% | 98,83% |
|
||||
| 5 | 98,42% | 99,02% |
|
||||
| 6 | 98,59% | 98,80% |
|
||||
| 7 | 98,72% | 98,93% |
|
||||
| 8 | 98,93% | 99,05% |
|
||||
| 9 | 98,96% | 99,05% |
|
||||
| 10 | 99,07% | 99,02% |
|
||||
|
||||
Validation accuracy sudah melampaui 98% sejak **epoch pertama** — sesuatu yang tidak pernah dicapai MLP bahkan setelah 10 epoch.
|
||||
|
||||
### 6.2 Perbandingan CNN vs MLP
|
||||
|
||||
| Model | Arsitektur | Jumlah Parameter | Test Accuracy | Waktu Training/Epoch |
|
||||
|---|---|---|---|---|
|
||||
| Model A (MLP) | 64 | 50.890 | 97,06% | 1,22 detik |
|
||||
| Model B (MLP) | 128 → 64 | 109.386 | 97,24% | 1,29 detik |
|
||||
| **Model C (MLP terbaik)** | 256 → 128 → 64 | 242.762 | **97,64%** | 2,50 detik |
|
||||
| **Model CNN** | Conv32 → Pool → Conv64 → Pool → Dense64 | **121.930** | **99,03%** | 16,56 detik |
|
||||
|
||||
### 6.3 Besar Improvement
|
||||
|
||||
| Metrik | MLP Terbaik | CNN | Perubahan |
|
||||
|---|---|---|---|
|
||||
| Test Accuracy | 97,64% | 99,03% | **+1,39 poin persentase** |
|
||||
| Test Loss | 0,0933 | 0,0297 | **−68,2%** |
|
||||
| Error Rate | 2,36% | 0,97% | **−58,9% (relatif)** |
|
||||
| Salah klasifikasi | 236 dari 10.000 | **97 dari 10.000** | 139 gambar lebih sedikit |
|
||||
| Jumlah Parameter | 242.762 | 121.930 | **−49,8% (separuh lebih sedikit)** |
|
||||
| Waktu/Epoch | 2,50 detik | 16,56 detik | +562% (lebih lambat) |
|
||||
|
||||
**Catatan penting mengenai interpretasi improvement:**
|
||||
|
||||
Kenaikan 1,39 poin persentase terlihat kecil, tetapi cara yang lebih tepat membacanya adalah lewat **error rate**. CNN memangkas hampir **59% kesalahan** yang masih dilakukan MLP. Pada skala industri, misalnya sistem pembacaan kode pos otomatis yang memproses 1 juta surat, perbedaan ini berarti sekitar **13.900 surat lebih sedikit yang salah baca**.
|
||||
|
||||
## 7. Analisis: Mengapa CNN Lebih Efektif untuk Data Gambar?
|
||||
|
||||
### 7.1 Mempertahankan Struktur Spasial
|
||||
|
||||
MLP meratakan gambar 28 × 28 menjadi vektor 784 angka. Setelah diratakan, piksel yang secara visual bersebelahan (misalnya piksel baris 1 kolom 28 dan baris 2 kolom 1) menjadi berjauhan, sementara model tidak punya cara mengetahui bahwa keduanya sebenarnya bertetangga. Informasi geometris gambar hilang sebelum training dimulai. CNN memproses gambar dalam bentuk 2D aslinya, sehingga pola lokal seperti tepi, sudut, dan lengkungan angka tetap utuh.
|
||||
|
||||
### 7.2 Local Receptive Field dan Parameter Sharing
|
||||
|
||||
Setiap filter konvolusi berukuran 3 × 3 hanya melihat area kecil gambar, lalu **digeser ke seluruh permukaan gambar dengan bobot yang sama**. Artinya, satu filter yang belajar mendeteksi garis diagonal dapat menemukan garis diagonal di mana pun posisinya.
|
||||
|
||||
Pada MLP, setiap posisi piksel punya bobotnya sendiri-sendiri, sehingga model harus mempelajari pola yang sama berulang kali untuk setiap posisi. Inilah alasan CNN mencapai akurasi lebih tinggi meskipun jumlah parameternya **separuh lebih sedikit** dari MLP terbaik.
|
||||
|
||||
### 7.3 Hierarki Fitur Otomatis
|
||||
|
||||
CNN membangun pemahaman secara bertingkat, mirip cara kerja sistem visual manusia:
|
||||
|
||||
``` text
|
||||
Conv layer 1 → fitur sederhana: tepi, garis, gradien terang-gelap
|
||||
↓
|
||||
Conv layer 2 → kombinasi fitur: lengkungan, sudut, lingkaran kecil
|
||||
↓
|
||||
Dense layer → konsep utuh: "ini bentuk angka 8"
|
||||
```
|
||||
|
||||
MLP tidak memiliki hierarki semacam ini karena semua piksel langsung dihubungkan ke semua neuron tanpa tahapan abstraksi spasial.
|
||||
|
||||
### 7.4 Translation Invariance melalui MaxPooling
|
||||
|
||||
`MaxPooling2D` mengambil nilai maksimum dari setiap area 2 × 2, sehingga pergeseran kecil posisi angka dalam gambar tidak banyak mengubah output. Model menjadi tahan terhadap variasi penulisan, misalnya angka yang ditulis sedikit lebih ke kiri atau lebih ke kanan. Pada MLP, pergeseran satu piksel saja mengubah seluruh vektor input dan dapat mengubah hasil prediksi.
|
||||
|
||||
### 7.5 Trade-off yang Perlu Dicatat
|
||||
|
||||
CNN unggul dalam akurasi dan efisiensi parameter, tetapi **waktu training per epoch jauh lebih lama** (16,56 detik vs 2,50 detik). Operasi konvolusi secara komputasi lebih mahal dibanding perkalian matriks pada Dense layer, karena filter harus digeser ke setiap posisi gambar. Perlu diingat bahwa pengukuran ini dilakukan pada CPU; pada GPU, selisih waktu ini mengecil drastis karena konvolusi sangat mudah diparalelkan.
|
||||
|
||||
## 8. Kesimpulan
|
||||
|
||||
1. Model CNN sederhana dengan dua blok `Conv2D` + `MaxPooling2D` berhasil mencapai **test accuracy 99,03%** pada MNIST.
|
||||
2. Dibandingkan MLP terbaik dari Tugas No. 1 (97,64%), CNN memberikan improvement **+1,39 poin persentase**, atau setara dengan **pengurangan error rate sebesar 58,9%**.
|
||||
3. CNN mencapai hasil tersebut dengan **jumlah parameter hampir separuh lebih sedikit**, membuktikan bahwa keunggulannya berasal dari *inductive bias* yang sesuai dengan sifat data gambar, bukan sekadar dari kapasitas model yang lebih besar.
|
||||
4. Keunggulan CNN untuk data gambar berakar pada empat hal: preservasi struktur spasial, parameter sharing, hierarki fitur otomatis, dan translation invariance.
|
||||
5. Konsekuensinya adalah waktu training per epoch yang lebih lama, sehingga pemilihan arsitektur tetap perlu mempertimbangkan ketersediaan sumber daya komputasi.
|
||||
|
||||
## 9. Teknologi
|
||||
|
||||
- Python 3.10 – 3.12
|
||||
- TensorFlow 2.x
|
||||
- Keras
|
||||
- NumPy
|
||||
- Matplotlib (untuk plot learning curve)
|
||||
- MNIST
|
||||
- **Visual Studio Code** (editor & runtime lokal)
|
||||
|
||||
## 10. Struktur Folder
|
||||
|
||||
```text
|
||||
tugas-mnist/
|
||||
├── .venv/ # virtual environment (tidak perlu di-commit)
|
||||
├── task4_cnn.py # program utama CNN
|
||||
├── README_No4_MNIST_CNN.md # dokumentasi ini
|
||||
└── requirements.txt
|
||||
```
|
||||
|
||||
Isi `requirements.txt`:
|
||||
|
||||
```text
|
||||
tensorflow>=2.16
|
||||
numpy
|
||||
matplotlib
|
||||
```
|
||||
|
||||
## 11. Cara Menjalankan di VS Code
|
||||
|
||||
### 11.1 Persiapan Awal (sekali saja)
|
||||
|
||||
**1. Install ekstensi yang dibutuhkan**
|
||||
|
||||
Buka panel Extensions di VS Code (`Ctrl+Shift+X`), lalu install:
|
||||
|
||||
- **Python** (Microsoft)
|
||||
- **Pylance** (Microsoft)
|
||||
- **Jupyter** (opsional, jika ingin menjalankan per-cell)
|
||||
|
||||
**2. Buka folder proyek**
|
||||
|
||||
`File → Open Folder…` lalu pilih folder `tugas-mnist`.
|
||||
|
||||
**3. Buat virtual environment**
|
||||
|
||||
Buka terminal terintegrasi dengan `` Ctrl+` `` lalu jalankan:
|
||||
|
||||
```bash
|
||||
# Windows
|
||||
python -m venv .venv
|
||||
.venv\Scripts\activate
|
||||
|
||||
# macOS / Linux
|
||||
python3 -m venv .venv
|
||||
source .venv/bin/activate
|
||||
```
|
||||
|
||||
**4. Pilih interpreter di VS Code**
|
||||
|
||||
Tekan `Ctrl+Shift+P` → ketik **Python: Select Interpreter** → pilih interpreter yang berada di dalam folder `.venv`. Langkah ini penting agar VS Code menjalankan kode memakai environment yang benar, bukan Python sistem.
|
||||
|
||||
**5. Install dependensi**
|
||||
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
Catatan: jika pemasangan TensorFlow terasa berat atau tidak memiliki GPU, gunakan versi CPU yang ukurannya lebih ringan:
|
||||
|
||||
```bash
|
||||
pip install tensorflow-cpu numpy matplotlib
|
||||
```
|
||||
|
||||
### 11.2 Menjalankan Program
|
||||
|
||||
Ada tiga cara, pilih salah satu:
|
||||
|
||||
| Cara | Langkah |
|
||||
|---|---|
|
||||
| Tombol Run | Buka `task4_cnn.py`, klik tombol **▶ Run Python File** di kanan atas |
|
||||
| Terminal | Jalankan `python task4_cnn.py` |
|
||||
| Debug | Tekan `F5`, pilih konfigurasi **Python File** |
|
||||
|
||||
### 11.3 Hal yang Perlu Diperhatikan
|
||||
|
||||
**Dataset akan diunduh otomatis pada eksekusi pertama.** `keras.datasets.mnist.load_data()` mengunduh berkas `mnist.npz` (± 11 MB) dan menyimpannya di:
|
||||
|
||||
```text
|
||||
Windows : C:\Users\<nama-user>\.keras\datasets\
|
||||
macOS/Linux : ~/.keras/datasets/
|
||||
```
|
||||
|
||||
Eksekusi berikutnya akan memakai cache tersebut, jadi tidak perlu koneksi internet lagi. Pastikan koneksi aktif saat menjalankan pertama kali.
|
||||
|
||||
**Waktu training di CPU.** Model CNN membutuhkan sekitar **15–20 detik per epoch** di CPU, sehingga total training 10 epoch memakan waktu sekitar **3 menit**. Ini wajar; jangan dikira program macet. Gunakan `verbose=2` agar progres tercetak rapi per epoch di terminal VS Code.
|
||||
|
||||
**Peringatan oneDNN/CUDA di awal output.** TensorFlow biasanya mencetak pesan seperti `Could not find cuda drivers` atau `oneDNN custom operations are on`. Pesan ini **normal dan bukan error** — artinya TensorFlow berjalan memakai CPU. Program tetap berjalan dengan benar. Jika ingin menyembunyikannya, tambahkan di baris paling atas program, sebelum `import tensorflow`:
|
||||
|
||||
```python
|
||||
import os
|
||||
os.environ["TF_CPP_MIN_LOG_LEVEL"] = "2"
|
||||
```
|
||||
|
||||
**Plot Matplotlib.** Saat dijalankan sebagai file `.py`, jendela plot akan terbuka terpisah dan program berhenti sampai jendela ditutup. Agar plot langsung tersimpan sebagai gambar tanpa mengganggu alur program, gunakan `plt.savefig("nama.png")` alih-alih `plt.show()`.
|
||||
|
||||
### 11.4 Langkah Pengerjaan Tugas
|
||||
|
||||
1. Jalankan program hingga selesai 10 epoch.
|
||||
2. Catat **test accuracy**, **jumlah parameter** (dari `model.summary()`), dan **waktu training per epoch**.
|
||||
3. Bandingkan hasilnya dengan tabel hasil MLP dari Tugas No. 1.
|
||||
4. Hitung improvement, baik dalam poin persentase maupun pengurangan error rate.
|
||||
5. Isi tabel pada bagian **Hasil Eksperimen** dengan angka aktual yang Anda peroleh.
|
||||
|
||||
Nilai akurasi yang Anda dapatkan mungkin berbeda tipis dari dokumen ini (biasanya di kisaran 98,9% – 99,2%) karena inisialisasi bobot bersifat acak. Untuk hasil yang dapat direproduksi, atur seed di awal program:
|
||||
|
||||
```python
|
||||
import numpy as np, tensorflow as tf
|
||||
tf.random.set_seed(42)
|
||||
np.random.seed(42)
|
||||
```
|
||||
@@ -1,3 +0,0 @@
|
||||
tensorflow>=2.16
|
||||
numpy
|
||||
matplotlib
|
||||
@@ -1,184 +0,0 @@
|
||||
"""
|
||||
Tugas No. 4 - Implementasi CNN sederhana untuk MNIST
|
||||
dan perbandingannya dengan MLP dari Tugas No. 1.
|
||||
|
||||
Cara menjalankan di VS Code:
|
||||
1. Aktifkan virtual environment
|
||||
2. pip install tensorflow numpy matplotlib
|
||||
3. Klik tombol Run, atau jalankan: python task4_cnn.py
|
||||
|
||||
Dataset akan diunduh otomatis saat pertama kali dijalankan
|
||||
dan disimpan di ~/.keras/datasets/
|
||||
"""
|
||||
|
||||
import os
|
||||
|
||||
# Sembunyikan log informasi TensorFlow (harus sebelum import tensorflow)
|
||||
os.environ["TF_CPP_MIN_LOG_LEVEL"] = "2"
|
||||
|
||||
import time
|
||||
|
||||
import matplotlib
|
||||
matplotlib.use("Agg") # simpan plot sebagai file, tidak membuka jendela
|
||||
import matplotlib.pyplot as plt
|
||||
import numpy as np
|
||||
import tensorflow as tf
|
||||
import keras
|
||||
from keras import layers
|
||||
|
||||
#Konfigurasi
|
||||
SEED = 42
|
||||
EPOCHS = 10
|
||||
BATCH_SIZE = 128
|
||||
VAL_SPLIT = 0.1
|
||||
|
||||
tf.random.set_seed(SEED)
|
||||
np.random.seed(SEED)
|
||||
|
||||
# Hasil MLP terbaik dari Tugas No. 1 (Model C: 256-128-64).
|
||||
# Ganti angka di bawah ini dengan hasil eksperimen Anda sendiri.
|
||||
MLP_ACCURACY = 0.9764
|
||||
MLP_PARAMS = 242_762
|
||||
MLP_EPOCH_TIME = 2.50
|
||||
|
||||
#1. Load dan preprocessing data
|
||||
print("Memuat dataset MNIST...")
|
||||
(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()
|
||||
|
||||
print(f"Data training : {x_train.shape[0]:,} gambar")
|
||||
print(f"Data testing : {x_test.shape[0]:,} gambar")
|
||||
print(f"Ukuran gambar : {x_train.shape[1]} x {x_train.shape[2]} piksel\n")
|
||||
|
||||
# ormalisasi nilai piksel dari 0-255 menjadi 0-1
|
||||
x_train = x_train.astype("float32") / 255.0
|
||||
x_test = x_test.astype("float32") / 255.0
|
||||
|
||||
#CNN membutuhkan dimensi channel: (N, 28, 28) -> (N, 28, 28, 1)
|
||||
#Berbeda dengan MLP yang meratakan gambar menjadi (N, 784)
|
||||
x_train = x_train.reshape(-1, 28, 28, 1)
|
||||
x_test = x_test.reshape(-1, 28, 28, 1)
|
||||
|
||||
#2. Bangun model CNN
|
||||
|
||||
model = keras.Sequential(
|
||||
name="cnn_mnist",
|
||||
layers=[
|
||||
layers.Input(shape=(28, 28, 1)),
|
||||
|
||||
# Blok konvolusi 1: deteksi fitur sederhana (tepi, garis)
|
||||
layers.Conv2D(32, (3, 3), activation="relu"),
|
||||
layers.MaxPooling2D((2, 2)),
|
||||
|
||||
# Blok konvolusi 2: gabungkan menjadi fitur lebih kompleks
|
||||
layers.Conv2D(64, (3, 3), activation="relu"),
|
||||
layers.MaxPooling2D((2, 2)),
|
||||
|
||||
# Klasifikasi
|
||||
layers.Flatten(),
|
||||
layers.Dense(64, activation="relu"),
|
||||
layers.Dropout(0.3),
|
||||
layers.Dense(10, activation="softmax"),
|
||||
],
|
||||
)
|
||||
|
||||
model.compile(
|
||||
optimizer="adam",
|
||||
loss="sparse_categorical_crossentropy",
|
||||
metrics=["accuracy"],
|
||||
)
|
||||
|
||||
model.summary()
|
||||
|
||||
# 3. Training dengan pencatatan waktu per epoch
|
||||
epoch_times = []
|
||||
|
||||
class TimeCallback(keras.callbacks.Callback):
|
||||
"""Mencatat durasi setiap epoch."""
|
||||
|
||||
def on_epoch_begin(self, epoch, logs=None):
|
||||
self._start = time.time()
|
||||
|
||||
def on_epoch_end(self, epoch, logs=None):
|
||||
epoch_times.append(time.time() - self._start)
|
||||
|
||||
|
||||
print("\nMemulai training CNN...")
|
||||
print("Perkiraan waktu: 15-20 detik per epoch di CPU (total sekitar 3 menit).\n")
|
||||
|
||||
history = model.fit(
|
||||
x_train,
|
||||
y_train,
|
||||
validation_split=VAL_SPLIT,
|
||||
epochs=EPOCHS,
|
||||
batch_size=BATCH_SIZE,
|
||||
verbose=2,
|
||||
callbacks=[TimeCallback()],
|
||||
)
|
||||
# 4. Evaluasi
|
||||
test_loss, test_acc = model.evaluate(x_test, y_test, verbose=0)
|
||||
n_params = model.count_params()
|
||||
avg_epoch_time = float(np.mean(epoch_times))
|
||||
|
||||
predictions = np.argmax(model.predict(x_test, verbose=0), axis=1)
|
||||
n_wrong = int(np.sum(predictions != y_test))
|
||||
|
||||
# 5. Perbandingan CNN vs MLP
|
||||
error_mlp = (1 - MLP_ACCURACY) * 100
|
||||
error_cnn = (1 - test_acc) * 100
|
||||
gain_point = (test_acc - MLP_ACCURACY) * 100
|
||||
error_reduction = (1 - error_cnn / error_mlp) * 100
|
||||
|
||||
print("\n" + "=" * 62)
|
||||
print("HASIL EKSPERIMEN CNN")
|
||||
print("=" * 62)
|
||||
print(f"Test accuracy : {test_acc * 100:.2f}%")
|
||||
print(f"Test loss : {test_loss:.4f}")
|
||||
print(f"Jumlah parameter : {n_params:,}")
|
||||
print(f"Waktu training/epoch : {avg_epoch_time:.2f} detik")
|
||||
print(f"Salah klasifikasi : {n_wrong} dari {len(y_test):,} gambar")
|
||||
|
||||
print("\n" + "=" * 62)
|
||||
print("PERBANDINGAN CNN vs MLP")
|
||||
print("=" * 62)
|
||||
print(f"{'Metrik':<24}{'MLP':>14}{'CNN':>14}")
|
||||
print("-" * 62)
|
||||
print(f"{'Test accuracy':<24}{MLP_ACCURACY * 100:>13.2f}%{test_acc * 100:>13.2f}%")
|
||||
print(f"{'Error rate':<24}{error_mlp:>13.2f}%{error_cnn:>13.2f}%")
|
||||
print(f"{'Jumlah parameter':<24}{MLP_PARAMS:>14,}{n_params:>14,}")
|
||||
print(f"{'Waktu/epoch (detik)':<24}{MLP_EPOCH_TIME:>14.2f}{avg_epoch_time:>14.2f}")
|
||||
print("-" * 62)
|
||||
print(f"Improvement akurasi : +{gain_point:.2f} poin persentase")
|
||||
print(f"Pengurangan error rate : {error_reduction:.1f}% (relatif)")
|
||||
print(f"Selisih parameter : {n_params - MLP_PARAMS:+,}")
|
||||
print("=" * 62)
|
||||
|
||||
# 6. Simpan grafik
|
||||
fig, axes = plt.subplots(1, 2, figsize=(12, 4.5))
|
||||
|
||||
epochs_range = range(1, EPOCHS + 1)
|
||||
axes[0].plot(epochs_range, history.history["accuracy"], label="Train accuracy")
|
||||
axes[0].plot(epochs_range, history.history["val_accuracy"], label="Validation accuracy")
|
||||
axes[0].set_title("Learning Curve CNN")
|
||||
axes[0].set_xlabel("Epoch")
|
||||
axes[0].set_ylabel("Accuracy")
|
||||
axes[0].legend()
|
||||
axes[0].grid(alpha=0.3)
|
||||
|
||||
axes[1].bar(
|
||||
["MLP terbaik", "CNN"],
|
||||
[MLP_ACCURACY * 100, test_acc * 100],
|
||||
color=["#55A868", "#C44E52"],
|
||||
)
|
||||
axes[1].set_ylim(96, 100)
|
||||
axes[1].set_ylabel("Test Accuracy (%)")
|
||||
axes[1].set_title("Perbandingan MLP vs CNN")
|
||||
for i, value in enumerate([MLP_ACCURACY * 100, test_acc * 100]):
|
||||
axes[1].text(i, value + 0.05, f"{value:.2f}%", ha="center")
|
||||
|
||||
plt.tight_layout()
|
||||
plt.savefig("hasil_cnn.png", dpi=130)
|
||||
print("\nGrafik disimpan sebagai: hasil_cnn.png")
|
||||
|
||||
# Simpan model terlatih agar tidak perlu training ulang
|
||||
model.save("model_cnn_mnist.keras")
|
||||
print("Model disimpan sebagai: model_cnn_mnist.keras")
|
||||
@@ -0,0 +1,148 @@
|
||||
"""
|
||||
Tugas No. 4 - Implementasi CNN sederhana untuk MNIST
|
||||
dan perbandingannya dengan MLP dari Tugas No. 1.
|
||||
"""
|
||||
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
|
||||
# Sembunyikan log informasi TensorFlow
|
||||
os.environ["TF_CPP_MIN_LOG_LEVEL"] = "2"
|
||||
|
||||
import matplotlib
|
||||
matplotlib.use("Agg") # simpan plot sebagai file
|
||||
import matplotlib.pyplot as plt
|
||||
import numpy as np
|
||||
import tensorflow as tf
|
||||
|
||||
# Tambahkan root directory ke path agar dapat import dari src
|
||||
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "..")))
|
||||
|
||||
from src.data_loader import load_mnist_for_cnn
|
||||
from src.models import create_cnn_mnist_model
|
||||
from src.utils import EpochTimerCallback, ensure_dir
|
||||
|
||||
|
||||
def main():
|
||||
# Konfigurasi
|
||||
SEED = 42
|
||||
EPOCHS = 10
|
||||
BATCH_SIZE = 128
|
||||
VAL_SPLIT = 0.1
|
||||
|
||||
tf.random.set_seed(SEED)
|
||||
np.random.seed(SEED)
|
||||
|
||||
# Hasil MLP terbaik dari Tugas No. 1 (Model C: 256-128-64)
|
||||
MLP_ACCURACY = 0.9764
|
||||
MLP_PARAMS = 242_762
|
||||
MLP_EPOCH_TIME = 2.50
|
||||
|
||||
# 1. Load dan preprocessing data
|
||||
print("Memuat dataset MNIST untuk CNN...")
|
||||
(x_train, y_train), (x_test, y_test) = load_mnist_for_cnn()
|
||||
|
||||
print(f"Data training : {x_train.shape[0]:,} gambar")
|
||||
print(f"Data testing : {x_test.shape[0]:,} gambar")
|
||||
print(f"Ukuran gambar : {x_train.shape[1]} x {x_train.shape[2]} piksel (channel={x_train.shape[3]})\n")
|
||||
|
||||
# 2. Bangun model CNN
|
||||
model = create_cnn_mnist_model()
|
||||
model.summary()
|
||||
|
||||
# 3. Training dengan pencatatan waktu per epoch
|
||||
timer_cb = EpochTimerCallback()
|
||||
|
||||
print("\nMemulai training CNN...")
|
||||
print("Perkiraan waktu: 15-20 detik per epoch di CPU (total sekitar 3 menit).\n")
|
||||
|
||||
history = model.fit(
|
||||
x_train,
|
||||
y_train,
|
||||
validation_split=VAL_SPLIT,
|
||||
epochs=EPOCHS,
|
||||
batch_size=BATCH_SIZE,
|
||||
verbose=2,
|
||||
callbacks=[timer_cb],
|
||||
)
|
||||
|
||||
# 4. Evaluasi
|
||||
test_loss, test_acc = model.evaluate(x_test, y_test, verbose=0)
|
||||
n_params = model.count_params()
|
||||
avg_epoch_time = float(np.mean(timer_cb.epoch_times)) if timer_cb.epoch_times else 0.0
|
||||
|
||||
predictions = np.argmax(model.predict(x_test, verbose=0), axis=1)
|
||||
n_wrong = int(np.sum(predictions != y_test))
|
||||
|
||||
# 5. Perbandingan CNN vs MLP
|
||||
error_mlp = (1 - MLP_ACCURACY) * 100
|
||||
error_cnn = (1 - test_acc) * 100
|
||||
gain_point = (test_acc - MLP_ACCURACY) * 100
|
||||
error_reduction = (1 - error_cnn / error_mlp) * 100
|
||||
|
||||
print("\n" + "=" * 62)
|
||||
print("HASIL EKSPERIMEN CNN")
|
||||
print("=" * 62)
|
||||
print(f"Test accuracy : {test_acc * 100:.2f}%")
|
||||
print(f"Test loss : {test_loss:.4f}")
|
||||
print(f"Jumlah parameter : {n_params:,}")
|
||||
print(f"Waktu training/epoch : {avg_epoch_time:.2f} detik")
|
||||
print(f"Salah klasifikasi : {n_wrong} dari {len(y_test):,} gambar")
|
||||
|
||||
print("\n" + "=" * 62)
|
||||
print("PERBANDINGAN CNN vs MLP")
|
||||
print("=" * 62)
|
||||
print(f"{'Metrik':<24}{'MLP':>14}{'CNN':>14}")
|
||||
print("-" * 62)
|
||||
print(f"{'Test accuracy':<24}{MLP_ACCURACY * 100:>13.2f}%{test_acc * 100:>13.2f}%")
|
||||
print(f"{'Error rate':<24}{error_mlp:>13.2f}%{error_cnn:>13.2f}%")
|
||||
print(f"{'Jumlah parameter':<24}{MLP_PARAMS:>14,}{n_params:>14,}")
|
||||
print(f"{'Waktu/epoch (detik)':<24}{MLP_EPOCH_TIME:>14.2f}{avg_epoch_time:>14.2f}")
|
||||
print("-" * 62)
|
||||
print(f"Improvement akurasi : +{gain_point:.2f} poin persentase")
|
||||
print(f"Pengurangan error rate : {error_reduction:.1f}% (relatif)")
|
||||
print(f"Selisih parameter : {n_params - MLP_PARAMS:+,}")
|
||||
print("=" * 62)
|
||||
|
||||
# 6. Simpan grafik ke folder results/02_cnn/figures
|
||||
results_fig_dir = os.path.join("results", "02_cnn", "figures")
|
||||
ensure_dir(results_fig_dir)
|
||||
fig_path = os.path.join(results_fig_dir, "hasil_cnn.png")
|
||||
|
||||
fig, axes = plt.subplots(1, 2, figsize=(12, 4.5))
|
||||
epochs_range = range(1, EPOCHS + 1)
|
||||
axes[0].plot(epochs_range, history.history["accuracy"], label="Train accuracy")
|
||||
axes[0].plot(epochs_range, history.history["val_accuracy"], label="Validation accuracy")
|
||||
axes[0].set_title("Learning Curve CNN")
|
||||
axes[0].set_xlabel("Epoch")
|
||||
axes[0].set_ylabel("Accuracy")
|
||||
axes[0].legend()
|
||||
axes[0].grid(alpha=0.3)
|
||||
|
||||
axes[1].bar(
|
||||
["MLP terbaik", "CNN"],
|
||||
[MLP_ACCURACY * 100, test_acc * 100],
|
||||
color=["#55A868", "#C44E52"],
|
||||
)
|
||||
axes[1].set_ylim(96, 100)
|
||||
axes[1].set_ylabel("Test Accuracy (%)")
|
||||
axes[1].set_title("Perbandingan MLP vs CNN")
|
||||
for i, value in enumerate([MLP_ACCURACY * 100, test_acc * 100]):
|
||||
axes[1].text(i, value + 0.05, f"{value:.2f}%", ha="center")
|
||||
|
||||
plt.tight_layout()
|
||||
plt.savefig(fig_path, dpi=130)
|
||||
print(f"\n[INFO] Grafik disimpan di: {fig_path}")
|
||||
|
||||
# 7. Simpan model terlatih ke folder results/02_cnn/models
|
||||
results_model_dir = os.path.join("results", "02_cnn", "models")
|
||||
ensure_dir(results_model_dir)
|
||||
model_path = os.path.join(results_model_dir, "model_cnn_mnist.keras")
|
||||
model.save(model_path)
|
||||
print(f"[INFO] Model disimpan di: {model_path}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
|
||||
@@ -0,0 +1,195 @@
|
||||
"""
|
||||
Tugas 1 (Week 2): Pipeline CNN MNIST Lengkap
|
||||
Memenuhi seluruh poin instruksi:
|
||||
(a) Model Summary
|
||||
(b) Loss & Accuracy curve training vs validation
|
||||
(c) Confusion Matrix pada test set
|
||||
(d) Perbandingan accuracy MLP vs CNN dan analisis efisiensi parameter
|
||||
"""
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
||||
# Sembunyikan log TensorFlow yang berlebihan
|
||||
os.environ["TF_CPP_MIN_LOG_LEVEL"] = "2"
|
||||
|
||||
import matplotlib
|
||||
matplotlib.use("Agg")
|
||||
import matplotlib.pyplot as plt
|
||||
import numpy as np
|
||||
import seaborn as sns
|
||||
from sklearn.metrics import classification_report, confusion_matrix
|
||||
import tensorflow as tf
|
||||
|
||||
# Tambahkan root directory ke path
|
||||
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "..")))
|
||||
|
||||
from src.data_loader import load_mnist_for_cnn
|
||||
from src.models import create_cnn_mnist_model
|
||||
from src.utils import EpochTimerCallback, ensure_dir
|
||||
|
||||
|
||||
def main():
|
||||
SEED = 42
|
||||
EPOCHS = 10
|
||||
BATCH_SIZE = 128
|
||||
VAL_SPLIT = 0.1
|
||||
|
||||
tf.random.set_seed(SEED)
|
||||
np.random.seed(SEED)
|
||||
|
||||
# Baseline MLP Terbaik (Model C dari Tugas 1 & 2)
|
||||
MLP_ACCURACY = 0.9824
|
||||
MLP_PARAMS = 244_554
|
||||
MLP_EPOCH_TIME = 2.50
|
||||
|
||||
print("=" * 70)
|
||||
print("TUGAS 1: PIPELINE CNN MNIST LENGKAP")
|
||||
print("=" * 70)
|
||||
|
||||
# 1. Load Data
|
||||
print("\n[1/5] Memuat dataset MNIST untuk CNN...")
|
||||
(x_train, y_train), (x_test, y_test) = load_mnist_for_cnn()
|
||||
print(f" -> Data training : {x_train.shape[0]:,} gambar ({x_train.shape[1]}x{x_train.shape[2]}x{x_train.shape[3]})")
|
||||
print(f" -> Data testing : {x_test.shape[0]:,} gambar")
|
||||
|
||||
# 2. Model Summary (Poin A)
|
||||
print("\n[2/5] Membangun Arsitektur CNN MNIST...")
|
||||
model = create_cnn_mnist_model()
|
||||
print("\n" + "-" * 70)
|
||||
print("(a) MODEL SUMMARY:")
|
||||
print("-" * 70)
|
||||
model.summary()
|
||||
|
||||
# 3. Training Model
|
||||
print("\n[3/5] Memulai proses training (10 Epochs)...")
|
||||
timer_cb = EpochTimerCallback()
|
||||
|
||||
history = model.fit(
|
||||
x_train,
|
||||
y_train,
|
||||
validation_split=VAL_SPLIT,
|
||||
epochs=EPOCHS,
|
||||
batch_size=BATCH_SIZE,
|
||||
verbose=1,
|
||||
callbacks=[timer_cb],
|
||||
)
|
||||
|
||||
avg_epoch_time = float(np.mean(timer_cb.epoch_times)) if timer_cb.epoch_times else 0.0
|
||||
|
||||
# 4. Evaluasi & Prediksi pada Test Set
|
||||
print("\n[4/5] Evaluasi pada Test Set (10,000 data)...")
|
||||
test_loss, test_acc = model.evaluate(x_test, y_test, verbose=0)
|
||||
y_pred_probs = model.predict(x_test, verbose=0)
|
||||
y_pred = np.argmax(y_pred_probs, axis=1)
|
||||
|
||||
print(f" -> Test Accuracy : {test_acc * 100:.2f}%")
|
||||
print(f" -> Test Loss : {test_loss:.4f}")
|
||||
|
||||
# 5. Visualisasi & Penyimpanan
|
||||
figures_dir = os.path.join("results", "02_cnn", "figures")
|
||||
ensure_dir(figures_dir)
|
||||
|
||||
# (b) Loss & Accuracy Curve
|
||||
print("\n[5/5] Membuat visualisasi...")
|
||||
curve_path = os.path.join(figures_dir, "loss_accuracy_cnn_lengkap.png")
|
||||
fig, axes = plt.subplots(1, 2, figsize=(13, 5))
|
||||
epochs_range = range(1, EPOCHS + 1)
|
||||
|
||||
# Loss Curve
|
||||
axes[0].plot(epochs_range, history.history["loss"], "b-o", label="Training Loss")
|
||||
axes[0].plot(epochs_range, history.history["val_loss"], "r--s", label="Validation Loss")
|
||||
axes[0].set_title("(b1) Loss Curve: Training vs Validation", fontsize=12, fontweight="bold")
|
||||
axes[0].set_xlabel("Epoch", fontsize=10)
|
||||
axes[0].set_ylabel("Cross-Entropy Loss", fontsize=10)
|
||||
axes[0].legend()
|
||||
axes[0].grid(True, alpha=0.3)
|
||||
|
||||
# Accuracy Curve
|
||||
axes[1].plot(epochs_range, history.history["accuracy"], "b-o", label="Training Accuracy")
|
||||
axes[1].plot(epochs_range, history.history["val_accuracy"], "r--s", label="Validation Accuracy")
|
||||
axes[1].set_title("(b2) Accuracy Curve: Training vs Validation", fontsize=12, fontweight="bold")
|
||||
axes[1].set_xlabel("Epoch", fontsize=10)
|
||||
axes[1].set_ylabel("Accuracy", fontsize=10)
|
||||
axes[1].legend()
|
||||
axes[1].grid(True, alpha=0.3)
|
||||
|
||||
plt.tight_layout()
|
||||
plt.savefig(curve_path, dpi=150)
|
||||
plt.close()
|
||||
print(f" -> (b) Kurva Loss & Accuracy tersimpan: {curve_path}")
|
||||
|
||||
# (c) Confusion Matrix pada Test Set
|
||||
cm_path = os.path.join(figures_dir, "confusion_matrix_cnn.png")
|
||||
cm = confusion_matrix(y_test, y_pred)
|
||||
|
||||
plt.figure(figsize=(9, 7.5))
|
||||
sns.heatmap(
|
||||
cm,
|
||||
annot=True,
|
||||
fmt="d",
|
||||
cmap="Blues",
|
||||
cbar=True,
|
||||
xticklabels=range(10),
|
||||
yticklabels=range(10),
|
||||
annot_kws={"size": 10}
|
||||
)
|
||||
plt.title("(c) Confusion Matrix CNN pada MNIST Test Set (10.000 Gambar)", fontsize=13, fontweight="bold", pad=12)
|
||||
plt.xlabel("Predicted Digit", fontsize=11, labelpad=8)
|
||||
plt.ylabel("True Digit", fontsize=11, labelpad=8)
|
||||
plt.tight_layout()
|
||||
plt.savefig(cm_path, dpi=150)
|
||||
plt.close()
|
||||
print(f" -> (c) Confusion Matrix tersimpan: {cm_path}")
|
||||
|
||||
# (d) Perbandingan Kuantitatif MLP vs CNN
|
||||
n_params_cnn = model.count_params()
|
||||
diff_acc = (test_acc - MLP_ACCURACY) * 100
|
||||
param_saving = (1 - n_params_cnn / MLP_PARAMS) * 100
|
||||
error_mlp = (1 - MLP_ACCURACY) * 100
|
||||
error_cnn = (1 - test_acc) * 100
|
||||
rel_error_red = (1 - error_cnn / error_mlp) * 100
|
||||
|
||||
print("\n" + "=" * 70)
|
||||
print("(d) PERBANDINGAN ACCURACY & EFISIENSI: MLP vs CNN")
|
||||
print("=" * 70)
|
||||
print(f"{'Metrik Evaluasi':<28}{'MLP Terbaik':>18}{'CNN Sederhana':>18}")
|
||||
print("-" * 70)
|
||||
print(f"{'Test Accuracy':<28}{MLP_ACCURACY * 100:>17.2f}%{test_acc * 100:>17.2f}%")
|
||||
print(f"{'Error Rate':<28}{error_mlp:>17.2f}%{error_cnn:>17.2f}%")
|
||||
print(f"{'Jumlah Parameter':<28}{MLP_PARAMS:>18,}{n_params_cnn:>18,}")
|
||||
print(f"{'Waktu Training / Epoch':<28}{MLP_EPOCH_TIME:>16.2f}s{avg_epoch_time:>16.2f}s")
|
||||
print("-" * 70)
|
||||
print(f"-> Improvement Akurasi : +{diff_acc:.2f}% poin persentase")
|
||||
print(f"-> Pengurangan Error : {rel_error_red:.1f}% penurunan error relatif")
|
||||
print(f"-> Efisiensi Parameter : CNN hemat {param_saving:.1f}% bobot dibanding MLP")
|
||||
print("=" * 70)
|
||||
|
||||
# Analisis Singkat Tertulis
|
||||
print("\n" + "=" * 70)
|
||||
print("ANALISIS: EFISIENSI PARAMETER & PERFORMA (MLP vs CNN)")
|
||||
print("=" * 70)
|
||||
print(
|
||||
f"1. Jumlah Parameter:\n"
|
||||
f" Model CNN memiliki {n_params_cnn:,} parameter, sedangkan MLP terbaik memiliki {MLP_PARAMS:,} parameter.\n"
|
||||
f" CNN menggunakan {param_saving:.1f}% parameter lebih sedikit (sekitar separuh dari MLP).\n\n"
|
||||
f"2. Mana yang Lebih Efisien?\n"
|
||||
f" CNN jauh lebih efisien. Meskipun memiliki parameter yang jauh lebih sedikit, CNN menghasilkan\n"
|
||||
f" akurasi {test_acc * 100:.2f}% (lebih tinggi dari MLP {MLP_ACCURACY * 100:.2f}%).\n\n"
|
||||
f"3. Mengapa demikian?\n"
|
||||
f" - Parameter Sharing: Filter konvolusi berukuran kecil (3x3) digunakan secara berulang di seluruh citra,\n"
|
||||
f" sehingga tidak membutuhkan koneksi bobot penuh antar setiap piksel seperti Dense layer pada MLP.\n"
|
||||
f" - Preservasi Spasial: CNN memproses data dalam representasi 2D asli (lebar x tinggi x channel),\n"
|
||||
f" menjaga hubungan antar-piksel tetangga, tepi, dan kontur angka tulisan tangan."
|
||||
)
|
||||
print("=" * 70 + "\n")
|
||||
|
||||
# Simpan bobot terbaru
|
||||
models_dir = os.path.join("results", "02_cnn", "models")
|
||||
ensure_dir(models_dir)
|
||||
model.save(os.path.join(models_dir, "model_cnn_mnist.keras"))
|
||||
print(f"[INFO] Model terbaru disimpan di: {os.path.join(models_dir, 'model_cnn_mnist.keras')}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,197 @@
|
||||
"""
|
||||
Tugas 2 (Week 2): Eksplorasi Kedalaman Arsitektur CNN
|
||||
Memodifikasi CNN dengan menambahkan satu blok Conv2D + MaxPooling ketiga.
|
||||
Mengevaluasi:
|
||||
1. Apakah model yang lebih dalam menghasilkan akurasi lebih tinggi?
|
||||
2. Bagaimana perbandingan waktu training per epoch?
|
||||
3. Catatan temuan dalam satu paragraf analitis.
|
||||
"""
|
||||
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
|
||||
# Sembunyikan log TensorFlow yang berlebihan
|
||||
os.environ["TF_CPP_MIN_LOG_LEVEL"] = "2"
|
||||
|
||||
import matplotlib
|
||||
matplotlib.use("Agg")
|
||||
import matplotlib.pyplot as plt
|
||||
import numpy as np
|
||||
import tensorflow as tf
|
||||
|
||||
# Tambahkan root directory ke path
|
||||
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "..")))
|
||||
|
||||
from src.data_loader import load_mnist_for_cnn
|
||||
from src.models import create_cnn_mnist_model, create_deep_cnn_mnist_model
|
||||
from src.utils import EpochTimerCallback, ensure_dir
|
||||
|
||||
|
||||
def main():
|
||||
SEED = 42
|
||||
EPOCHS = 10
|
||||
BATCH_SIZE = 128
|
||||
VAL_SPLIT = 0.1
|
||||
|
||||
tf.random.set_seed(SEED)
|
||||
np.random.seed(SEED)
|
||||
|
||||
print("=" * 75)
|
||||
print("TUGAS 2: EKSPLORASI KEDALAMAN ARSITEKTUR CNN (2 BLOK vs 3 BLOK)")
|
||||
print("=" * 75)
|
||||
|
||||
# 1. Load Data
|
||||
print("\nMemuat dataset MNIST...")
|
||||
(x_train, y_train), (x_test, y_test) = load_mnist_for_cnn()
|
||||
|
||||
architectures = [
|
||||
{
|
||||
"name": "CNN 2 Blok (Baseline)",
|
||||
"builder": create_cnn_mnist_model,
|
||||
"desc": "Conv(32) + MaxPool -> Conv(64) + MaxPool -> Dense(64)"
|
||||
},
|
||||
{
|
||||
"name": "CNN 3 Blok (Lebih Dalam)",
|
||||
"builder": create_deep_cnn_mnist_model,
|
||||
"desc": "Conv(32) + MaxPool -> Conv(64) + MaxPool -> Conv(128) + MaxPool -> Dense(64)"
|
||||
}
|
||||
]
|
||||
|
||||
results = []
|
||||
histories = {}
|
||||
|
||||
for arch in architectures:
|
||||
model_name = arch["name"]
|
||||
print("\n" + "=" * 75)
|
||||
print(f"PELATIHAN: {model_name}")
|
||||
print(f"Konfigurasi: {arch['desc']}")
|
||||
print("=" * 75)
|
||||
|
||||
model = arch["builder"]()
|
||||
model.summary()
|
||||
|
||||
timer_cb = EpochTimerCallback()
|
||||
start_train_time = time.time()
|
||||
|
||||
history = model.fit(
|
||||
x_train,
|
||||
y_train,
|
||||
validation_split=VAL_SPLIT,
|
||||
epochs=EPOCHS,
|
||||
batch_size=BATCH_SIZE,
|
||||
verbose=1,
|
||||
callbacks=[timer_cb]
|
||||
)
|
||||
|
||||
total_train_time = time.time() - start_train_time
|
||||
avg_epoch_time = float(np.mean(timer_cb.epoch_times)) if timer_cb.epoch_times else 0.0
|
||||
|
||||
# Evaluasi pada test set
|
||||
test_loss, test_acc = model.evaluate(x_test, y_test, verbose=0)
|
||||
n_params = model.count_params()
|
||||
|
||||
results.append({
|
||||
"name": model_name,
|
||||
"params": n_params,
|
||||
"test_acc": test_acc * 100,
|
||||
"test_loss": test_loss,
|
||||
"epoch_time": avg_epoch_time,
|
||||
"total_time": total_train_time,
|
||||
"model_obj": model
|
||||
})
|
||||
|
||||
histories[model_name] = history
|
||||
|
||||
# Simpan model 3 blok
|
||||
models_dir = os.path.join("results", "02_cnn", "models")
|
||||
ensure_dir(models_dir)
|
||||
results[1]["model_obj"].save(os.path.join(models_dir, "model_cnn_3blocks.keras"))
|
||||
|
||||
# Tabel Perbandingan Kuantitatif
|
||||
print("\n\n" + "=" * 75)
|
||||
print("HASIL PERBANDINGAN: KEDALAMAN CNN PADA MNIST")
|
||||
print("=" * 75)
|
||||
print(f"{'Arsitektur Model':<28}{'Parameter':>14}{'Test Acc':>14}{'Waktu/Epoch':>18}")
|
||||
print("-" * 75)
|
||||
for r in results:
|
||||
print(f"{r['name']:<28}{r['params']:>14,}{r['test_acc']:>13.2f}%{r['epoch_time']:>16.2f}s")
|
||||
print("-" * 75)
|
||||
|
||||
diff_acc = results[1]["test_acc"] - results[0]["test_acc"]
|
||||
time_increase = ((results[1]["epoch_time"] - results[0]["epoch_time"]) / results[0]["epoch_time"]) * 100
|
||||
param_diff = results[1]["params"] - results[0]["params"]
|
||||
|
||||
print(f"Selisih Akurasi (3 Blok vs 2 Blok) : {diff_acc:+.2f}% poin")
|
||||
print(f"Perubahan Waktu Training / Epoch : {time_increase:+.1f}%")
|
||||
print(f"Selisih Jumlah Parameter : {param_diff:+,} parameter")
|
||||
print("=" * 75)
|
||||
|
||||
# Visualisasi
|
||||
figures_dir = os.path.join("results", "02_cnn", "figures")
|
||||
ensure_dir(figures_dir)
|
||||
fig_path = os.path.join(figures_dir, "perbandingan_kedalaman_cnn.png")
|
||||
|
||||
fig, axes = plt.subplots(1, 2, figsize=(13, 5))
|
||||
epochs_range = range(1, EPOCHS + 1)
|
||||
|
||||
# Plot Accuracy Perbandingan
|
||||
for name, hist in histories.items():
|
||||
axes[0].plot(epochs_range, hist.history["val_accuracy"], label=f"{name} (Val Acc)", linewidth=2)
|
||||
axes[0].set_title("Validation Accuracy per Epoch", fontsize=12, fontweight="bold")
|
||||
axes[0].set_xlabel("Epoch", fontsize=10)
|
||||
axes[0].set_ylabel("Accuracy", fontsize=10)
|
||||
axes[0].legend()
|
||||
axes[0].grid(True, alpha=0.3)
|
||||
|
||||
# Bar chart perbandingan Waktu vs Akurasi
|
||||
labels = [r["name"] for r in results]
|
||||
accs = [r["test_acc"] for r in results]
|
||||
times = [r["epoch_time"] for r in results]
|
||||
|
||||
x = np.arange(len(labels))
|
||||
width = 0.35
|
||||
|
||||
ax2 = axes[1].twinx()
|
||||
p1 = axes[1].bar(x - width/2, accs, width, label="Test Accuracy (%)", color="#3498db")
|
||||
p2 = ax2.bar(x + width/2, times, width, label="Waktu / Epoch (s)", color="#e67e22")
|
||||
|
||||
axes[1].set_ylabel("Accuracy (%)", color="#3498db", fontsize=10)
|
||||
ax2.set_ylabel("Detik per Epoch", color="#e67e22", fontsize=10)
|
||||
axes[1].set_xticks(x)
|
||||
axes[1].set_xticklabels(labels, fontsize=10)
|
||||
axes[1].set_ylim(97, 100)
|
||||
axes[1].set_title("Trade-off: Test Accuracy vs Waktu Training", fontsize=12, fontweight="bold")
|
||||
|
||||
# Nilai di atas bar
|
||||
for bar in p1:
|
||||
axes[1].text(bar.get_x() + bar.get_width()/2, bar.get_height() + 0.05, f"{bar.get_height():.2f}%", ha='center', va='bottom', fontsize=9)
|
||||
for bar in p2:
|
||||
ax2.text(bar.get_x() + bar.get_width()/2, bar.get_height() + 0.2, f"{bar.get_height():.2f}s", ha='center', va='bottom', fontsize=9)
|
||||
|
||||
plt.tight_layout()
|
||||
plt.savefig(fig_path, dpi=150)
|
||||
plt.close()
|
||||
print(f"\n[INFO] Grafik perbandingan tersimpan di: {fig_path}")
|
||||
|
||||
# Catatan Analisis Satu Paragraf (Sesuai Instruksi Tugas 2)
|
||||
print("\n" + "=" * 75)
|
||||
print("CATATAN TEMUAN (SATU PARAGRAF ANALISIS):")
|
||||
print("=" * 75)
|
||||
paragraf = (
|
||||
f"Penambahan blok ketiga (Conv2D 128 filter + MaxPooling2D) pada citra MNIST (28x28 piksel) "
|
||||
f"menghasilkan akurasi test sebesar {results[1]['test_acc']:.2f}% dibandingkan dengan {results[0]['test_acc']:.2f}% "
|
||||
f"pada baseline 2 blok (selisih {diff_acc:+.2f}% poin), namun meningkatkan waktu komputasi training "
|
||||
f"sebesar {time_increase:+.1f}% per epoch ({results[1]['epoch_time']:.2f}s vs {results[0]['epoch_time']:.2f}s) "
|
||||
f"serta menambah kompleksitas parameter. Pada dataset sederhana dengan resolusi spasial kecil seperti MNIST, "
|
||||
f"tiga kali operasi pooling mereduksi resolusi spasial representasi secara drastis (hingga 3x3 piksel), "
|
||||
f"sehingga arsitektur yang lebih dalam mengalami fenomena diminishing returns di mana penambahan layer "
|
||||
f"tidak memberikan peningkatan performa yang sebanding dengan biaya komputasi yang dikeluarkan; "
|
||||
f"oleh karena itu, arsitektur CNN 2 blok tetap menjadi pilihan paling optimal dan efisien untuk MNIST."
|
||||
)
|
||||
print(paragraf)
|
||||
print("=" * 75 + "\n")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,302 @@
|
||||
"""
|
||||
Tugas Anggota 2 (Week 2): CNN Evaluation + Visualization
|
||||
=========================================================
|
||||
PENTING - KONSISTENSI KONFIGURASI DENGAN ANGGOTA 1:
|
||||
Skrip ini TIDAK melatih ulang model dari nol. Sebagai gantinya, skrip ini
|
||||
memuat bobot model CNN yang sudah dilatih oleh Anggota 1
|
||||
(results/models/model_cnn_mnist.keras) pada 05_cnn_full_pipeline.py.
|
||||
|
||||
Ini adalah cara paling aman untuk menjamin "tidak ada konfigurasi berbeda"
|
||||
antara Anggota 1 dan Anggota 2, karena:
|
||||
- Dataset yang sama -> load_mnist_for_cnn() (src/data_loader.py)
|
||||
- Preprocessing yang sama -> normalisasi 0-1, reshape (28, 28, 1)
|
||||
- Epoch yang sama -> 10 epoch (bobot hasil training Anggota 1)
|
||||
- Batch size yang sama -> 128 (dipakai saat training oleh Anggota 1)
|
||||
- Optimizer yang sama -> Adam (tersimpan di dalam file .keras)
|
||||
|
||||
Konstanta di bawah HANYA didokumentasikan ulang (bukan dipakai untuk
|
||||
melatih ulang) agar terlihat eksplisit bahwa konfigurasi cocok dengan
|
||||
skrip Anggota 1.
|
||||
|
||||
Cakupan tugas (sesuai skema kelompok):
|
||||
(1) Evaluasi menyeluruh pada test set (classification report per kelas)
|
||||
(2) Visualisasi confusion matrix (raw & ternormalisasi/persentase)
|
||||
(3) Visualisasi akurasi per kelas (digit mana yang paling sulit)
|
||||
(4) Visualisasi distribusi confidence prediksi (benar vs salah)
|
||||
(5) Visualisasi contoh gambar yang salah diklasifikasikan
|
||||
"""
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
||||
# Sembunyikan log TensorFlow yang berlebihan
|
||||
os.environ["TF_CPP_MIN_LOG_LEVEL"] = "2"
|
||||
|
||||
import matplotlib
|
||||
matplotlib.use("Agg")
|
||||
import matplotlib.pyplot as plt
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import seaborn as sns
|
||||
from sklearn.metrics import classification_report, confusion_matrix
|
||||
import tensorflow as tf
|
||||
from tensorflow import keras
|
||||
|
||||
# Tambahkan root directory ke path
|
||||
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "..")))
|
||||
|
||||
from src.data_loader import load_mnist_for_cnn
|
||||
from src.utils import ensure_dir
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# KONFIGURASI (harus identik dengan Anggota 1 / 05_cnn_full_pipeline.py)
|
||||
# ---------------------------------------------------------------------------
|
||||
SEED = 42
|
||||
EPOCHS = 10 # epoch training model (didokumentasikan, tidak dipakai ulang)
|
||||
BATCH_SIZE = 128 # batch size training model (didokumentasikan, tidak dipakai ulang)
|
||||
OPTIMIZER = "adam" # optimizer training model (tersimpan di file .keras)
|
||||
|
||||
MODEL_PATH = os.path.join("results", "02_cnn", "models", "model_cnn_mnist.keras")
|
||||
FIGURES_DIR = os.path.join("results", "02_cnn", "figures")
|
||||
REPORTS_DIR = os.path.join("results", "02_cnn", "reports")
|
||||
|
||||
CLASS_NAMES = [str(i) for i in range(10)]
|
||||
|
||||
|
||||
def main():
|
||||
tf.random.set_seed(SEED)
|
||||
np.random.seed(SEED)
|
||||
|
||||
ensure_dir(FIGURES_DIR)
|
||||
ensure_dir(REPORTS_DIR)
|
||||
|
||||
print("=" * 75)
|
||||
print("TUGAS ANGGOTA 2: CNN EVALUATION + VISUALIZATION")
|
||||
print("=" * 75)
|
||||
print(
|
||||
f"[KONFIGURASI - selaras dengan Anggota 1] "
|
||||
f"seed={SEED} | epochs={EPOCHS} | batch_size={BATCH_SIZE} | optimizer={OPTIMIZER}"
|
||||
)
|
||||
|
||||
# 1. Load Data (dataset & preprocessing sama persis dengan Anggota 1)
|
||||
print("\n[1/6] Memuat dataset MNIST (load_mnist_for_cnn - sama dengan Anggota 1)...")
|
||||
(x_train, y_train), (x_test, y_test) = load_mnist_for_cnn()
|
||||
print(f" -> Data testing : {x_test.shape[0]:,} gambar {x_test.shape[1:]}")
|
||||
|
||||
# 2. Load Model Terlatih dari Anggota 1
|
||||
print(f"\n[2/6] Memuat model terlatih dari: {MODEL_PATH}")
|
||||
if not os.path.exists(MODEL_PATH):
|
||||
raise FileNotFoundError(
|
||||
f"Model tidak ditemukan di '{MODEL_PATH}'. "
|
||||
f"Jalankan dahulu tasks/cnn/05_cnn_full_pipeline.py (skrip Anggota 1)."
|
||||
)
|
||||
model = keras.models.load_model(MODEL_PATH)
|
||||
print(f" -> Model '{model.name}' berhasil dimuat ({model.count_params():,} parameter).")
|
||||
|
||||
# 3. Evaluasi Umum pada Test Set
|
||||
print("\n[3/6] Mengevaluasi model pada test set...")
|
||||
test_loss, test_acc = model.evaluate(x_test, y_test, verbose=0)
|
||||
y_pred_probs = model.predict(x_test, verbose=0)
|
||||
y_pred = np.argmax(y_pred_probs, axis=1)
|
||||
y_confidence = np.max(y_pred_probs, axis=1)
|
||||
is_correct = (y_pred == y_test)
|
||||
|
||||
print(f" -> Test Accuracy : {test_acc * 100:.2f}%")
|
||||
print(f" -> Test Loss : {test_loss:.4f}")
|
||||
print(f" -> Total Salah : {np.sum(~is_correct):,} dari {len(y_test):,} gambar")
|
||||
|
||||
# -----------------------------------------------------------------
|
||||
# (1) Classification Report per Kelas
|
||||
# -----------------------------------------------------------------
|
||||
print("\n[4/6] Membuat classification report per kelas...")
|
||||
report_dict = classification_report(
|
||||
y_test, y_pred, target_names=CLASS_NAMES, digits=4, output_dict=True
|
||||
)
|
||||
report_text = classification_report(y_test, y_pred, target_names=CLASS_NAMES, digits=4)
|
||||
print("\n" + "-" * 75)
|
||||
print("CLASSIFICATION REPORT (Precision / Recall / F1-Score per Digit):")
|
||||
print("-" * 75)
|
||||
print(report_text)
|
||||
|
||||
report_txt_path = os.path.join(REPORTS_DIR, "classification_report_cnn.txt")
|
||||
with open(report_txt_path, "w") as f:
|
||||
f.write("CLASSIFICATION REPORT - CNN MNIST (Anggota 2: CNN Evaluation)\n")
|
||||
f.write(f"Test Accuracy: {test_acc * 100:.2f}% | Test Loss: {test_loss:.4f}\n")
|
||||
f.write("=" * 65 + "\n")
|
||||
f.write(report_text)
|
||||
print(f" -> Laporan teks tersimpan: {report_txt_path}")
|
||||
|
||||
report_df = pd.DataFrame(report_dict).transpose()
|
||||
report_csv_path = os.path.join(REPORTS_DIR, "classification_report_cnn.csv")
|
||||
report_df.to_csv(report_csv_path)
|
||||
print(f" -> Laporan CSV tersimpan : {report_csv_path}")
|
||||
|
||||
# Visualisasi precision/recall/f1 per kelas (bar chart berkelompok)
|
||||
per_class_df = report_df.loc[CLASS_NAMES, ["precision", "recall", "f1-score"]]
|
||||
fig, ax = plt.subplots(figsize=(11, 5))
|
||||
per_class_df.plot(kind="bar", ax=ax, width=0.75,
|
||||
color=["#3498db", "#2ecc71", "#e74c3c"])
|
||||
ax.set_title("Precision, Recall, dan F1-Score per Digit (CNN)", fontsize=13, fontweight="bold")
|
||||
ax.set_xlabel("Digit", fontsize=11)
|
||||
ax.set_ylabel("Skor", fontsize=11)
|
||||
ax.set_ylim(0.90, 1.005)
|
||||
ax.set_xticklabels(CLASS_NAMES, rotation=0)
|
||||
ax.legend(["Precision", "Recall", "F1-Score"], loc="lower right")
|
||||
ax.grid(True, axis="y", alpha=0.3)
|
||||
plt.tight_layout()
|
||||
metrics_bar_path = os.path.join(FIGURES_DIR, "per_class_metrics_cnn.png")
|
||||
plt.savefig(metrics_bar_path, dpi=150)
|
||||
plt.close()
|
||||
print(f" -> Grafik metrik per kelas tersimpan: {metrics_bar_path}")
|
||||
|
||||
# -----------------------------------------------------------------
|
||||
# (2) Confusion Matrix: Raw Count & Ternormalisasi (%)
|
||||
# -----------------------------------------------------------------
|
||||
print("\n[5/6] Membuat visualisasi confusion matrix (raw & normalisasi)...")
|
||||
cm = confusion_matrix(y_test, y_pred)
|
||||
cm_norm = cm.astype("float") / cm.sum(axis=1, keepdims=True) * 100
|
||||
|
||||
fig, axes = plt.subplots(1, 2, figsize=(17, 7))
|
||||
sns.heatmap(cm, annot=True, fmt="d", cmap="Blues", cbar=True,
|
||||
xticklabels=CLASS_NAMES, yticklabels=CLASS_NAMES,
|
||||
annot_kws={"size": 9}, ax=axes[0])
|
||||
axes[0].set_title("Confusion Matrix (Jumlah Gambar)", fontsize=12, fontweight="bold")
|
||||
axes[0].set_xlabel("Predicted Digit")
|
||||
axes[0].set_ylabel("True Digit")
|
||||
|
||||
sns.heatmap(cm_norm, annot=True, fmt=".1f", cmap="Oranges", cbar=True,
|
||||
xticklabels=CLASS_NAMES, yticklabels=CLASS_NAMES,
|
||||
annot_kws={"size": 9}, ax=axes[1])
|
||||
axes[1].set_title("Confusion Matrix (Persentase per Baris / Recall %)", fontsize=12, fontweight="bold")
|
||||
axes[1].set_xlabel("Predicted Digit")
|
||||
axes[1].set_ylabel("True Digit")
|
||||
|
||||
plt.suptitle("Evaluasi CNN pada MNIST Test Set (10.000 Gambar)", fontsize=14, fontweight="bold", y=1.02)
|
||||
plt.tight_layout()
|
||||
cm_dual_path = os.path.join(FIGURES_DIR, "confusion_matrix_cnn_detail.png")
|
||||
plt.savefig(cm_dual_path, dpi=150, bbox_inches="tight")
|
||||
plt.close()
|
||||
print(f" -> Confusion matrix detail tersimpan: {cm_dual_path}")
|
||||
|
||||
# -----------------------------------------------------------------
|
||||
# (3) Akurasi per Kelas (diagonal confusion matrix ternormalisasi)
|
||||
# -----------------------------------------------------------------
|
||||
per_class_acc = np.diag(cm_norm)
|
||||
fig, ax = plt.subplots(figsize=(10, 5))
|
||||
colors = ["#e74c3c" if a < per_class_acc.mean() else "#2ecc71" for a in per_class_acc]
|
||||
bars = ax.bar(CLASS_NAMES, per_class_acc, color=colors)
|
||||
ax.axhline(per_class_acc.mean(), color="gray", linestyle="--", linewidth=1.5,
|
||||
label=f"Rata-rata: {per_class_acc.mean():.2f}%")
|
||||
for bar, acc in zip(bars, per_class_acc):
|
||||
ax.text(bar.get_x() + bar.get_width() / 2, bar.get_height() - 1.2,
|
||||
f"{acc:.1f}%", ha="center", va="top", fontsize=9, color="white", fontweight="bold")
|
||||
ax.set_title("Akurasi per Digit pada Test Set (CNN)", fontsize=13, fontweight="bold")
|
||||
ax.set_xlabel("Digit")
|
||||
ax.set_ylabel("Akurasi (%)")
|
||||
ax.set_ylim(min(per_class_acc) - 5, 100.5)
|
||||
ax.legend()
|
||||
ax.grid(True, axis="y", alpha=0.3)
|
||||
plt.tight_layout()
|
||||
per_class_acc_path = os.path.join(FIGURES_DIR, "per_class_accuracy_cnn.png")
|
||||
plt.savefig(per_class_acc_path, dpi=150)
|
||||
plt.close()
|
||||
print(f" -> Grafik akurasi per kelas tersimpan: {per_class_acc_path}")
|
||||
|
||||
hardest_idx = np.argsort(per_class_acc)[:3]
|
||||
print(" -> 3 digit paling sulit dikenali:")
|
||||
for idx in hardest_idx:
|
||||
print(f" Digit {idx}: akurasi {per_class_acc[idx]:.2f}%")
|
||||
|
||||
# -----------------------------------------------------------------
|
||||
# (4) Distribusi Confidence Prediksi (Benar vs Salah)
|
||||
# -----------------------------------------------------------------
|
||||
print("\n[6/6] Membuat visualisasi distribusi confidence & sampel kesalahan...")
|
||||
fig, ax = plt.subplots(figsize=(9, 5.5))
|
||||
ax.hist(y_confidence[is_correct], bins=30, alpha=0.65, color="#2ecc71",
|
||||
label=f"Prediksi Benar (n={np.sum(is_correct):,})", density=True)
|
||||
ax.hist(y_confidence[~is_correct], bins=30, alpha=0.75, color="#e74c3c",
|
||||
label=f"Prediksi Salah (n={np.sum(~is_correct):,})", density=True)
|
||||
ax.set_title("Distribusi Confidence Prediksi Model CNN", fontsize=13, fontweight="bold")
|
||||
ax.set_xlabel("Confidence (Probabilitas Softmax Maksimum)")
|
||||
ax.set_ylabel("Kepadatan (Density)")
|
||||
ax.legend()
|
||||
ax.grid(True, alpha=0.3)
|
||||
plt.tight_layout()
|
||||
confidence_path = os.path.join(FIGURES_DIR, "confidence_distribution_cnn.png")
|
||||
plt.savefig(confidence_path, dpi=150)
|
||||
plt.close()
|
||||
print(f" -> Distribusi confidence tersimpan: {confidence_path}")
|
||||
|
||||
avg_conf_correct = y_confidence[is_correct].mean() * 100
|
||||
avg_conf_wrong = y_confidence[~is_correct].mean() * 100
|
||||
print(f" -> Rata-rata confidence saat BENAR : {avg_conf_correct:.2f}%")
|
||||
print(f" -> Rata-rata confidence saat SALAH : {avg_conf_wrong:.2f}%")
|
||||
|
||||
# -----------------------------------------------------------------
|
||||
# (5) Contoh Gambar yang Salah Diklasifikasikan
|
||||
# -----------------------------------------------------------------
|
||||
wrong_idx = np.where(~is_correct)[0]
|
||||
rng = np.random.default_rng(SEED)
|
||||
n_show = min(25, len(wrong_idx))
|
||||
sample_idx = rng.choice(wrong_idx, size=n_show, replace=False)
|
||||
|
||||
fig, axes = plt.subplots(5, 5, figsize=(11, 12))
|
||||
for ax, idx in zip(axes.flat, sample_idx):
|
||||
ax.imshow(x_test[idx].reshape(28, 28), cmap="gray")
|
||||
conf = y_confidence[idx] * 100
|
||||
ax.set_title(f"True: {y_test[idx]} | Pred: {y_pred[idx]}\nConf: {conf:.1f}%",
|
||||
fontsize=9, color="#c0392b")
|
||||
ax.axis("off")
|
||||
for ax in axes.flat[n_show:]:
|
||||
ax.axis("off")
|
||||
plt.suptitle(f"Contoh {n_show} Gambar Salah Klasifikasi dari {len(wrong_idx)} Total Kesalahan",
|
||||
fontsize=13, fontweight="bold", y=1.0)
|
||||
plt.tight_layout()
|
||||
misclassified_path = os.path.join(FIGURES_DIR, "misclassified_samples_cnn.png")
|
||||
plt.savefig(misclassified_path, dpi=150, bbox_inches="tight")
|
||||
plt.close()
|
||||
print(f" -> Contoh gambar salah klasifikasi tersimpan: {misclassified_path}")
|
||||
|
||||
# -----------------------------------------------------------------
|
||||
# Ringkasan Akhir
|
||||
# -----------------------------------------------------------------
|
||||
summary_path = os.path.join(REPORTS_DIR, "evaluation_summary_cnn.csv")
|
||||
summary_df = pd.DataFrame([{
|
||||
"test_accuracy_pct": round(test_acc * 100, 4),
|
||||
"test_loss": round(test_loss, 4),
|
||||
"total_errors": int(np.sum(~is_correct)),
|
||||
"total_samples": int(len(y_test)),
|
||||
"avg_confidence_correct_pct": round(avg_conf_correct, 4),
|
||||
"avg_confidence_wrong_pct": round(avg_conf_wrong, 4),
|
||||
"hardest_digit_1": int(hardest_idx[0]),
|
||||
"hardest_digit_1_acc_pct": round(per_class_acc[hardest_idx[0]], 4),
|
||||
"hardest_digit_2": int(hardest_idx[1]),
|
||||
"hardest_digit_2_acc_pct": round(per_class_acc[hardest_idx[1]], 4),
|
||||
}])
|
||||
summary_df.to_csv(summary_path, index=False)
|
||||
|
||||
print("\n" + "=" * 75)
|
||||
print("RINGKASAN EVALUASI CNN (ANGGOTA 2)")
|
||||
print("=" * 75)
|
||||
print(f"Test Accuracy : {test_acc * 100:.2f}%")
|
||||
print(f"Test Loss : {test_loss:.4f}")
|
||||
print(f"Total Kesalahan : {np.sum(~is_correct):,} / {len(y_test):,}")
|
||||
print(f"Confidence saat Benar : {avg_conf_correct:.2f}%")
|
||||
print(f"Confidence saat Salah : {avg_conf_wrong:.2f}%")
|
||||
print(f"Digit Paling Sulit : {hardest_idx[0]} ({per_class_acc[hardest_idx[0]]:.2f}%), "
|
||||
f"{hardest_idx[1]} ({per_class_acc[hardest_idx[1]]:.2f}%)")
|
||||
print("-" * 75)
|
||||
print("File yang dihasilkan:")
|
||||
print(f" 1. {metrics_bar_path}")
|
||||
print(f" 2. {cm_dual_path}")
|
||||
print(f" 3. {per_class_acc_path}")
|
||||
print(f" 4. {confidence_path}")
|
||||
print(f" 5. {misclassified_path}")
|
||||
print(f" 6. {report_txt_path}")
|
||||
print(f" 7. {report_csv_path}")
|
||||
print(f" 8. {summary_path}")
|
||||
print("=" * 75 + "\n")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,137 @@
|
||||
import sys
|
||||
import os
|
||||
import time
|
||||
import pandas as pd
|
||||
import matplotlib.pyplot as plt
|
||||
|
||||
# Tambahkan root directory ke path agar dapat import dari src
|
||||
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "..")))
|
||||
|
||||
from src.data_loader import load_mnist_for_mlp
|
||||
from src.models import create_mlp_model
|
||||
|
||||
|
||||
def main():
|
||||
print("=" * 60)
|
||||
print("TUGAS 1: MEMUAT DATASET MNIST & EKSPERIMEN MLP")
|
||||
print("=" * 60)
|
||||
|
||||
# 1. Load Data
|
||||
(x_train, y_train), (x_test, y_test) = load_mnist_for_mlp()
|
||||
print(f"Data training : {x_train.shape}")
|
||||
print(f"Data testing : {x_test.shape}")
|
||||
|
||||
# 2. Konfigurasi 3 Model MLP
|
||||
models_config = {
|
||||
"Model A - 1 Hidden Layer": [64],
|
||||
"Model B - 2 Hidden Layer": [128, 64],
|
||||
"Model C - 3 Hidden Layer": [256, 128, 64]
|
||||
}
|
||||
|
||||
results = []
|
||||
EPOCHS = 10
|
||||
BATCH_SIZE = 128
|
||||
|
||||
# 3. Training & Evaluasi
|
||||
for model_name, architecture in models_config.items():
|
||||
print("\n" + "=" * 60)
|
||||
print(model_name)
|
||||
print("=" * 60)
|
||||
|
||||
model = create_mlp_model(architecture)
|
||||
model.summary()
|
||||
|
||||
print("\nMemulai training...")
|
||||
start_time = time.time()
|
||||
|
||||
model.fit(
|
||||
x_train,
|
||||
y_train,
|
||||
epochs=EPOCHS,
|
||||
batch_size=BATCH_SIZE,
|
||||
validation_split=0.1,
|
||||
verbose=1
|
||||
)
|
||||
|
||||
total_time = time.time() - start_time
|
||||
average_time = total_time / EPOCHS
|
||||
|
||||
test_loss, test_accuracy = model.evaluate(x_test, y_test, verbose=0)
|
||||
total_parameters = model.count_params()
|
||||
|
||||
results.append({
|
||||
"model": model_name,
|
||||
"parameters": total_parameters,
|
||||
"accuracy": test_accuracy * 100,
|
||||
"time_per_epoch": average_time
|
||||
})
|
||||
|
||||
print("\nHasil:")
|
||||
print(f"Test Accuracy : {test_accuracy * 100:.2f}%")
|
||||
print(f"Jumlah Parameter : {total_parameters:,}")
|
||||
print(f"Waktu/Epoch : {average_time:.2f} detik")
|
||||
|
||||
# 4. Ringkasan Perbandingan
|
||||
print("\n\n" + "=" * 80)
|
||||
print("HASIL PERBANDINGAN 3 MODEL MLP")
|
||||
print("=" * 80)
|
||||
print(f"{'Model':<30}{'Parameter':>15}{'Accuracy':>15}{'Time/Epoch':>15}")
|
||||
print("-" * 80)
|
||||
|
||||
for res in results:
|
||||
print(
|
||||
f"{res['model']:<30}"
|
||||
f"{res['parameters']:>15,}"
|
||||
f"{res['accuracy']:>14.2f}%"
|
||||
f"{res['time_per_epoch']:>14.2f}s"
|
||||
)
|
||||
|
||||
best_model = max(results, key=lambda x: x["accuracy"])
|
||||
print("\n" + "=" * 60)
|
||||
print("MODEL DENGAN TEST ACCURACY TERTINGGI")
|
||||
print("=" * 60)
|
||||
print(f"Model : {best_model['model']}")
|
||||
print(f"Test Accuracy : {best_model['accuracy']:.2f}%")
|
||||
print(f"Jumlah Parameter : {best_model['parameters']:,}")
|
||||
print(f"Waktu Training/Epoch: {best_model['time_per_epoch']:.2f} detik")
|
||||
|
||||
# 5. Simpan Hasil ke Folder results/01_mlp
|
||||
REPORTS_DIR = os.path.join("results", "01_mlp", "reports")
|
||||
FIGURES_DIR = os.path.join("results", "01_mlp", "figures")
|
||||
os.makedirs(REPORTS_DIR, exist_ok=True)
|
||||
os.makedirs(FIGURES_DIR, exist_ok=True)
|
||||
|
||||
df_results = pd.DataFrame(results)
|
||||
csv_path = os.path.join(REPORTS_DIR, "perbandingan_mlp.csv")
|
||||
df_results.to_csv(csv_path, index=False)
|
||||
print(f"\n[OK] Ringkasan CSV disimpan ke : {csv_path}")
|
||||
|
||||
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(12, 5))
|
||||
names = ["Model A\n(1 Layer)", "Model B\n(2 Layer)", "Model C\n(3 Layer)"]
|
||||
accs = [r["accuracy"] for r in results]
|
||||
bars1 = ax1.bar(names, accs, color=["#4C72B0", "#55A868", "#C44E52"])
|
||||
ax1.set_title("Perbandingan Test Accuracy (%)", fontsize=12, fontweight="bold")
|
||||
ax1.set_ylabel("Accuracy (%)")
|
||||
ax1.set_ylim([max(0, min(accs) - 2), 100])
|
||||
for bar in bars1:
|
||||
yval = bar.get_height()
|
||||
ax1.text(bar.get_x() + bar.get_width()/2, yval + 0.2, f"{yval:.2f}%", ha='center', va='bottom')
|
||||
|
||||
times = [r["time_per_epoch"] for r in results]
|
||||
bars2 = ax2.bar(names, times, color=["#4C72B0", "#55A868", "#C44E52"])
|
||||
ax2.set_title("Waktu Training per Epoch (detik)", fontsize=12, fontweight="bold")
|
||||
ax2.set_ylabel("Detik / Epoch")
|
||||
for bar in bars2:
|
||||
yval = bar.get_height()
|
||||
ax2.text(bar.get_x() + bar.get_width()/2, yval + 0.05, f"{yval:.2f}s", ha='center', va='bottom')
|
||||
|
||||
plt.tight_layout()
|
||||
fig_path = os.path.join(FIGURES_DIR, "perbandingan_3_model_mlp.png")
|
||||
plt.savefig(fig_path, dpi=130)
|
||||
plt.close()
|
||||
print(f"[OK] Grafik perbandingan disimpan ke: {fig_path}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
|
||||
@@ -0,0 +1,93 @@
|
||||
import sys
|
||||
import os
|
||||
import matplotlib
|
||||
matplotlib.use("Agg")
|
||||
import matplotlib.pyplot as plt
|
||||
|
||||
# Tambahkan root directory ke path agar dapat import dari src
|
||||
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "..")))
|
||||
|
||||
from src.data_loader import load_mnist_for_mlp
|
||||
from src.models import create_mlp_model, create_regularized_mlp_model
|
||||
from src.utils import ensure_dir
|
||||
|
||||
|
||||
def main():
|
||||
print("=" * 60)
|
||||
print("TUGAS 2: EKSPERIMEN REGULARISASI (BASELINE vs BATCHNORM + DROPOUT)")
|
||||
print("=" * 60)
|
||||
|
||||
# 1. Load Data
|
||||
(x_train, y_train), (x_test, y_test) = load_mnist_for_mlp()
|
||||
|
||||
EPOCHS = 20
|
||||
BATCH_SIZE = 128
|
||||
|
||||
# 2. Baseline Model (Without Regularization)
|
||||
print("\n" + "=" * 60)
|
||||
print("TRAINING BASELINE MODEL (WITHOUT REGULARIZATION)")
|
||||
print("=" * 60)
|
||||
model_base = create_mlp_model([256, 128, 64])
|
||||
history_base = model_base.fit(
|
||||
x_train, y_train,
|
||||
epochs=EPOCHS,
|
||||
batch_size=BATCH_SIZE,
|
||||
validation_split=0.1,
|
||||
verbose=1
|
||||
)
|
||||
|
||||
# 3. Regularized Model (BatchNorm + Dropout 0.3)
|
||||
print("\n" + "=" * 60)
|
||||
print("TRAINING REGULARIZED MODEL (WITH BATCHNORM + DROPOUT)")
|
||||
print("=" * 60)
|
||||
model_reg = create_regularized_mlp_model(hidden_layers=[256, 128, 64], dropout_rate=0.3)
|
||||
history_reg = model_reg.fit(
|
||||
x_train, y_train,
|
||||
epochs=EPOCHS,
|
||||
batch_size=BATCH_SIZE,
|
||||
validation_split=0.1,
|
||||
verbose=1
|
||||
)
|
||||
|
||||
# 4. Save Model Artifact
|
||||
results_model_dir = os.path.join("results", "01_mlp", "models")
|
||||
ensure_dir(results_model_dir)
|
||||
model_reg_path = os.path.join(results_model_dir, "model_c_regularized.keras")
|
||||
model_reg.save(model_reg_path)
|
||||
print(f"\n[INFO] Model teregularisasi disimpan di: {model_reg_path}")
|
||||
|
||||
# 5. Plot Comparison Curves
|
||||
results_fig_dir = os.path.join("results", "01_mlp", "figures")
|
||||
ensure_dir(results_fig_dir)
|
||||
fig_path = os.path.join(results_fig_dir, "loss_regularization.png")
|
||||
|
||||
plt.figure(figsize=(14, 5))
|
||||
|
||||
# Subplot 1: Baseline
|
||||
plt.subplot(1, 2, 1)
|
||||
plt.plot(history_base.history['loss'], label='Training Loss', color='blue')
|
||||
plt.plot(history_base.history['val_loss'], label='Validation Loss', color='orange', linestyle='--')
|
||||
plt.title('Model C Without Regularization')
|
||||
plt.xlabel('Epoch')
|
||||
plt.ylabel('Loss')
|
||||
plt.legend()
|
||||
plt.grid(True, alpha=0.3)
|
||||
|
||||
# Subplot 2: Regularized
|
||||
plt.subplot(1, 2, 2)
|
||||
plt.plot(history_reg.history['loss'], label='Training Loss', color='blue')
|
||||
plt.plot(history_reg.history['val_loss'], label='Validation Loss', color='green', linestyle='--')
|
||||
plt.title('Model C With BatchNorm + Dropout(0.3)')
|
||||
plt.xlabel('Epoch')
|
||||
plt.ylabel('Loss')
|
||||
plt.legend()
|
||||
plt.grid(True, alpha=0.3)
|
||||
|
||||
plt.tight_layout()
|
||||
plt.savefig(fig_path, dpi=130)
|
||||
print(f"[INFO] Grafik perbandingan loss disimpan di: {fig_path}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
|
||||
@@ -0,0 +1,67 @@
|
||||
"""
|
||||
Tugas 3: Contoh Implementasi Berbagai Arsitektur Model Sesuai Karakteristik Data
|
||||
"""
|
||||
|
||||
import tensorflow as tf
|
||||
from tensorflow import keras
|
||||
from tensorflow.keras.layers import Conv2D, Dense, Embedding, Flatten, LSTM, MaxPooling2D
|
||||
from tensorflow.keras.models import Sequential
|
||||
from xgboost import XGBClassifier
|
||||
|
||||
|
||||
def build_models():
|
||||
print("=" * 60)
|
||||
print("TUGAS 3: CONTOH ARSITEKTUR MODEL SELECTION")
|
||||
print("=" * 60)
|
||||
|
||||
# (a) Prediksi Stok Barang dari Deret Waktu (LSTM)
|
||||
# Input shape: (samples, time_steps=12, features=1)
|
||||
model_a = Sequential([
|
||||
LSTM(32, input_shape=(12, 1)),
|
||||
Dense(1)
|
||||
], name="LSTM_Stock_Forecasting")
|
||||
model_a.compile(optimizer='adam', loss='mse')
|
||||
print("\n[Case A] Model LSTM untuk Prediksi Stok Barang (Time Series):")
|
||||
model_a.summary()
|
||||
|
||||
# (b) Klasifikasi Foto Produk Rusak vs Normal (CNN)
|
||||
# Input shape contoh citra RGB 128x128
|
||||
model_b = Sequential([
|
||||
Conv2D(32, (3, 3), activation='relu', input_shape=(128, 128, 3)),
|
||||
MaxPooling2D((2, 2)),
|
||||
Flatten(),
|
||||
Dense(64, activation='relu'),
|
||||
Dense(1, activation='sigmoid') # binary: rusak vs normal
|
||||
], name="CNN_Defect_Classification")
|
||||
model_b.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])
|
||||
print("\n[Case B] Model CNN untuk Klasifikasi Foto Produk (Computer Vision):")
|
||||
model_b.summary()
|
||||
|
||||
# (c) Analisis Sentimen Ulasan Play Store (LSTM + Embedding)
|
||||
vocab_size = 5000
|
||||
max_length = 100
|
||||
model_c = Sequential([
|
||||
Embedding(input_dim=vocab_size, output_dim=64, input_length=max_length),
|
||||
LSTM(64),
|
||||
Dense(32, activation='relu'),
|
||||
Dense(1, activation='sigmoid') # binary: positif vs negatif
|
||||
], name="LSTM_Sentiment_Analysis")
|
||||
model_c.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])
|
||||
print("\n[Case C] Model LSTM + Embedding untuk Analisis Sentimen (NLP):")
|
||||
model_c.summary()
|
||||
|
||||
# (d) Prediksi Churn Pelanggan Tabular (ML Klasik - XGBoost)
|
||||
# X_train memiliki dimensi (n_samples, 15)
|
||||
model_d = XGBClassifier(
|
||||
n_estimators=100,
|
||||
max_depth=4,
|
||||
learning_rate=0.1,
|
||||
random_state=42
|
||||
)
|
||||
print("\n[Case D] Model XGBoost untuk Prediksi Churn (Tabular 15 fitur):")
|
||||
print(model_d)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
build_models()
|
||||
|
||||
@@ -0,0 +1,288 @@
|
||||
import os
|
||||
import re
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import matplotlib.pyplot as plt
|
||||
|
||||
from datasets import load_dataset
|
||||
|
||||
from sklearn.model_selection import train_test_split
|
||||
from sklearn.metrics import (
|
||||
classification_report,
|
||||
confusion_matrix,
|
||||
ConfusionMatrixDisplay
|
||||
)
|
||||
|
||||
import tensorflow as tf
|
||||
from tensorflow import keras
|
||||
from tensorflow.keras import layers
|
||||
from tensorflow.keras.callbacks import EarlyStopping
|
||||
from tensorflow.keras.preprocessing.text import Tokenizer # type: ignore
|
||||
from tensorflow.keras.preprocessing.sequence import pad_sequences
|
||||
|
||||
# ============================================================
|
||||
# KONFIGURASI DIREKTORI PENYIMPANAN
|
||||
# ============================================================
|
||||
|
||||
VOCAB_SIZE = 10000
|
||||
MAX_LEN = 60
|
||||
EMBED_DIM = 64
|
||||
|
||||
EPOCHS = 20
|
||||
BATCH_SIZE = 64
|
||||
RANDOM_STATE = 42
|
||||
|
||||
FIGURES_DIR = os.path.join("results", "03_nlp_sentiment", "figures")
|
||||
REPORTS_DIR = os.path.join("results", "03_nlp_sentiment", "reports")
|
||||
MODELS_DIR = os.path.join("results", "03_nlp_sentiment", "models")
|
||||
|
||||
os.makedirs(FIGURES_DIR, exist_ok=True)
|
||||
os.makedirs(REPORTS_DIR, exist_ok=True)
|
||||
os.makedirs(MODELS_DIR, exist_ok=True)
|
||||
|
||||
|
||||
# ============================================================
|
||||
# SEL 7 — LOAD DATASET TOKOPEDIA
|
||||
# ============================================================
|
||||
|
||||
print("=" * 60)
|
||||
print("SEL 7 — LOAD DATASET")
|
||||
print("=" * 60)
|
||||
|
||||
dataset = load_dataset(
|
||||
"farhamu/tokopedia-product-reviews-2019"
|
||||
)
|
||||
|
||||
df = dataset["train"].to_pandas()
|
||||
|
||||
# Buang rating 3
|
||||
df = df[df["rating"] != 3].copy()
|
||||
|
||||
# Rating 4-5 = positif (1), Rating 1-2 = negatif (0)
|
||||
df["label"] = (df["rating"] >= 4).astype(int)
|
||||
|
||||
# Rename text menjadi teks
|
||||
df = df.rename(
|
||||
columns={"text": "teks"}
|
||||
)[["teks", "label"]].reset_index(drop=True)
|
||||
|
||||
print(df.head(5))
|
||||
print(f"\nShape : {df.shape}")
|
||||
print(f"Kolom : {df.columns.tolist()}")
|
||||
print(f"\nDistribusi label:\n{df['label'].value_counts().sort_index()}")
|
||||
print(f"\nNull : {df.isnull().sum().sum()}")
|
||||
|
||||
|
||||
# ============================================================
|
||||
# DISTRIBUSI LABEL
|
||||
# ============================================================
|
||||
|
||||
label_counts = df["label"].value_counts().sort_index()
|
||||
|
||||
plt.figure(figsize=(7, 5))
|
||||
plt.bar(
|
||||
["Negatif", "Positif"],
|
||||
[label_counts.get(0, 0), label_counts.get(1, 0)]
|
||||
)
|
||||
plt.title("Distribusi Label Sentimen Tokopedia")
|
||||
plt.xlabel("Sentimen")
|
||||
plt.ylabel("Jumlah Review")
|
||||
plt.tight_layout()
|
||||
|
||||
plt.savefig(f"{FIGURES_DIR}/distribusi_label_sentimen.png")
|
||||
plt.close()
|
||||
|
||||
|
||||
# ============================================================
|
||||
# CEK PANJANG TEKS
|
||||
# ============================================================
|
||||
|
||||
df["panjang"] = df["teks"].apply(lambda x: len(str(x).split()))
|
||||
print(f"\nPanjang teks:\n{df['panjang'].describe()}")
|
||||
|
||||
|
||||
# ============================================================
|
||||
# SEL 8 — TEXT PREPROCESSING
|
||||
# ============================================================
|
||||
|
||||
print("\n" + "=" * 60)
|
||||
print("SEL 8 — TEXT PREPROCESSING")
|
||||
print("=" * 60)
|
||||
|
||||
def clean_text(teks):
|
||||
teks = str(teks).lower()
|
||||
teks = re.sub(r"[^a-z\s]", "", teks)
|
||||
teks = re.sub(r"\s+", " ", teks).strip()
|
||||
return teks
|
||||
|
||||
df["teks_bersih"] = df["teks"].apply(clean_text)
|
||||
|
||||
# Train/Test Split
|
||||
X = np.array(df["teks_bersih"].tolist())
|
||||
y = np.array(df["label"].tolist(), dtype=int)
|
||||
|
||||
X_tr, X_te, y_tr, y_te = train_test_split(
|
||||
X, y,
|
||||
test_size=0.2,
|
||||
random_state=RANDOM_STATE,
|
||||
stratify=y
|
||||
)
|
||||
|
||||
print(f"Training data : {len(X_tr)}")
|
||||
print(f"Testing data : {len(X_te)}")
|
||||
|
||||
# Tokenizer
|
||||
tokenizer = Tokenizer(num_words=VOCAB_SIZE, oov_token="<OOV>")
|
||||
tokenizer.fit_on_texts(X_tr)
|
||||
|
||||
# Text -> Integer Sequence
|
||||
X_tr_seq = tokenizer.texts_to_sequences(X_tr)
|
||||
X_te_seq = tokenizer.texts_to_sequences(X_te)
|
||||
|
||||
# Padding
|
||||
X_tr_pad = pad_sequences(X_tr_seq, maxlen=MAX_LEN, padding="post")
|
||||
X_te_pad = pad_sequences(X_te_seq, maxlen=MAX_LEN, padding="post")
|
||||
|
||||
print(f"\nX_tr_pad shape : {X_tr_pad.shape}")
|
||||
print(f"X_te_pad shape : {X_te_pad.shape}")
|
||||
print(f"Vocab size : {len(tokenizer.word_index)}")
|
||||
|
||||
|
||||
# ============================================================
|
||||
# SEL 9 — MODEL GAP
|
||||
# ============================================================
|
||||
|
||||
print("\n" + "=" * 60)
|
||||
print("SEL 9 — MODEL GAP")
|
||||
print("=" * 60)
|
||||
|
||||
sent_model = keras.Sequential([
|
||||
layers.Embedding(input_dim=VOCAB_SIZE, output_dim=EMBED_DIM),
|
||||
layers.GlobalAveragePooling1D(),
|
||||
layers.Dense(64, activation="relu"),
|
||||
layers.Dropout(0.4),
|
||||
layers.Dense(1, activation="sigmoid")
|
||||
], name="Sentimen_GAP")
|
||||
|
||||
sent_model.compile(
|
||||
optimizer="adam",
|
||||
loss="binary_crossentropy",
|
||||
metrics=["accuracy"]
|
||||
)
|
||||
|
||||
sent_model.summary()
|
||||
|
||||
|
||||
# ============================================================
|
||||
# TRAINING
|
||||
# ============================================================
|
||||
|
||||
print("\n" + "=" * 60)
|
||||
print("TRAINING MODEL GAP")
|
||||
print("=" * 60)
|
||||
|
||||
early_stop = EarlyStopping(
|
||||
monitor="val_loss",
|
||||
patience=4,
|
||||
restore_best_weights=True
|
||||
)
|
||||
|
||||
history_sent = sent_model.fit(
|
||||
X_tr_pad, y_tr,
|
||||
epochs=EPOCHS,
|
||||
batch_size=BATCH_SIZE,
|
||||
validation_split=0.15,
|
||||
callbacks=[early_stop],
|
||||
verbose=1
|
||||
)
|
||||
|
||||
|
||||
# ============================================================
|
||||
# TEST ACCURACY & HISTORY PLOTS
|
||||
# ============================================================
|
||||
|
||||
loss_s, acc_s = sent_model.evaluate(X_te_pad, y_te, verbose=0)
|
||||
print(f"\nSentimen GAP → Test accuracy: {acc_s:.4f}")
|
||||
|
||||
# Plot Accuracy & Loss
|
||||
plt.figure(figsize=(8, 5))
|
||||
plt.plot(history_sent.history["accuracy"], label="Training Accuracy")
|
||||
plt.plot(history_sent.history["val_accuracy"], label="Validation Accuracy")
|
||||
plt.title("Training vs Validation Accuracy — GAP")
|
||||
plt.xlabel("Epoch")
|
||||
plt.ylabel("Accuracy")
|
||||
plt.legend()
|
||||
plt.tight_layout()
|
||||
plt.savefig(f"{FIGURES_DIR}/histori_gap_sentimen.png")
|
||||
plt.close()
|
||||
|
||||
|
||||
# ============================================================
|
||||
# SEL 11 — EVALUATION
|
||||
# ============================================================
|
||||
|
||||
print("\n" + "=" * 60)
|
||||
print("EVALUATION")
|
||||
print("=" * 60)
|
||||
|
||||
y_pred_prob = sent_model.predict(X_te_pad, verbose=0)
|
||||
y_pred = (y_pred_prob > 0.5).astype("int").flatten()
|
||||
|
||||
report = classification_report(
|
||||
y_te, y_pred,
|
||||
target_names=["Negatif (0)", "Positif (1)"]
|
||||
)
|
||||
|
||||
print("\nClassification Report:")
|
||||
print(report)
|
||||
|
||||
# Simpan Classification Report Txt
|
||||
with open(f"{REPORTS_DIR}/classification_report_gap_sentimen.txt", "w", encoding="utf-8") as f:
|
||||
f.write(report)
|
||||
|
||||
# Confusion Matrix Plot
|
||||
cm = confusion_matrix(y_te, y_pred)
|
||||
disp = ConfusionMatrixDisplay(confusion_matrix=cm, display_labels=["Negatif", "Positif"])
|
||||
fig, ax = plt.subplots(figsize=(6, 6))
|
||||
disp.plot(ax=ax, cmap="Blues")
|
||||
plt.title("Confusion Matrix — Model Sentimen GAP")
|
||||
plt.tight_layout()
|
||||
plt.savefig(f"{FIGURES_DIR}/confusion_matrix_gap_sentimen.png")
|
||||
plt.close()
|
||||
|
||||
|
||||
# ============================================================
|
||||
# SEL 12 — INFERENCE (PREDIKSI 5 ULASAN)
|
||||
# ============================================================
|
||||
|
||||
def prediksi_sentimen(teks, model, tokenizer, maxlen):
|
||||
teks_bersih = clean_text(teks)
|
||||
seq = tokenizer.texts_to_sequences([teks_bersih])
|
||||
padded = pad_sequences(seq, maxlen=maxlen, padding="post")
|
||||
prob = model.predict(padded, verbose=0)[0][0]
|
||||
label = "Positif" if prob > 0.5 else "Negatif"
|
||||
return label, prob
|
||||
|
||||
print("\n" + "=" * 60)
|
||||
print("PREDIKSI 5 ULASAN")
|
||||
print("=" * 60)
|
||||
|
||||
ulasan_uji = [
|
||||
"Produk berkualitas tinggi, pengiriman cepat, packing aman!",
|
||||
"Kecewa banget, barang datang rusak dan customer service tidak responsif",
|
||||
"Lumayan sih, tapi harganya kemahalan untuk kualitas segitu",
|
||||
"Tidak sesuai deskripsi sama sekali, mau return tapi ribet",
|
||||
"Barang bagus, sesuai dengan foto dan penjual sangat ramah"
|
||||
]
|
||||
|
||||
for i, ulasan in enumerate(ulasan_uji, start=1):
|
||||
label, prob = prediksi_sentimen(ulasan, sent_model, tokenizer, MAX_LEN)
|
||||
print(f"\n{i}. {ulasan}\n Prediksi : {label}\n Confidence: {prob:.3f}")
|
||||
|
||||
|
||||
# ============================================================
|
||||
# SIMPAN MODEL
|
||||
# ============================================================
|
||||
|
||||
sent_model.save(f"{MODELS_DIR}/model_gap_sentimen.keras")
|
||||
print("\nModel GAP berhasil disimpan di folder results/models/")
|
||||