<div style="text-align: right; background-color: #f5f5f5; padding: 4px 12px; border-radius: 5px; display: inline-block; float: right; font-size: 1.4rem;"> Last Updated：2025/08/15 </div> # Jupyter Lab 操作範例說明 [TOC] <br> 本範例使用政府資料開放平台提供的 [臺北市氣候資料](https://data.gov.tw/dataset/136718)，請先下載原始資料檔案（CSV 格式）：**`a05000201-213521473.csv`**。  - **範例說明**：示範如何利用 LSTM 模型進行時間序列預測。 - **預測目標**：根據歷史氣候數據（如溫度等）預測未來溫度，或其他您指定的目標欄位。 <br> ## 透過傳輸檔案節點上傳 `.csv` 檔案 請使用 Windows PowerShell 將下載好的 CSV 檔案上傳至創進一號（Forerunner 1）來進行傳輸。 <div style="border: 1.5px solid #B7B7B7; padding: 1rem 1rem; border-radius: 5px; color: #333;"> 🔗 <strong>詳細教學請參閱</strong> <a href="https://man.twcc.ai/@f1-manual/transport_ip#%E6%AA%94%E6%A1%88%E5%82%B3%E8%BC%B8%E7%AF%80%E9%BB%9E%E6%93%8D%E4%BD%9C%E6%B5%81%E7%A8%8B" target="_blank" style="color: #007ACC;"> 檔案傳輸節點操作流程 </a> </div> <br> ``` # 透過傳輸檔案節點 f1-dtn01.nchc.org.tw 或是 f1-dtn02.nchc.org.tw 上傳資料 # user 改成創進一號的username # put 指令上傳你的檔案（檔案路徑） PS C:\Users\XXX\Downloads> sftp user@f1-dtn01.nchc.org.tw Connected to f1-dtn01.nchc.org.tw. sftp> put a05000201-213521473.csv Uploading a05000201-213521473.csv to /home/user/a05000201-213521473.csv a05000201-213521473.csv 100% 1970 29.6KB/s 00:00 sftp> ``` <br> --- <br> ## 預先建立自訂 Miniconda Environment :::warning <i class="fa fa-bullhorn" aria-hidden="true"></i> **備註**： 在 Open OnDemand（OOD）環境中使用 Jupyter Notebook 或 Jupyter Lab 時，需注意以下限制與操作建議： - **OOD 執行於計算節點，無法連接外部網路進行 Python 套件的即時安裝**，需事先於登入節點（Login Node）安裝 Miniconda 並建立虛擬環境，並註冊為 Jupyter Kernel 使用。 - 由於 OOD 的使用限制，每位使用者**同一時間僅能啟動一個 Interactive Session**，若需啟動新的資源，請先手動刪除既有的 Session，以釋出資源。 ➤ 詳細說明請參考 [Application – Jupyter Lab](https://man.twcc.ai/@f1-manual/App_Jupyter_Lab) ::: **1. 登入 Login Node** 請使用 SSH 工具登入 F1 的登入節點 `f1-ilgn01.nchc.org.tw` 或 `f1-ilgn02.nchc.org.tw`。 <div style="border: 1.5px solid #B7B7B7; padding: 1rem 1rem; border-radius: 5px; color: #333;"> 🔗 <strong>詳細教學請參閱</strong> <a href="https://man.twcc.ai/@AI-Pilot/ByTJJNVz1g" target="_blank" style="color: #007ACC;"> 主機登入／登出教學連結 </a> </div> <br> **2. 安裝 Miniconda3** ``` [user@ilgn01 ~]$ mkdir -p ~/miniconda3 $ wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda3/miniconda.sh $ bash ~/miniconda3/miniconda.sh -b -u -p ~/miniconda3 $ rm -rf ~/miniconda3/miniconda.sh $ ~/miniconda3/bin/conda init bash $ ~/miniconda3/bin/conda init zsh ``` <font color="red">➤ **執行完**</font> `conda init` <font color="red">**後請重新登入 Login Node 登入節點，確保環境變數正確更新。**</font> <br> **3. 建立虛擬環境並安裝 Jupyter 相關工具** - 使用下列指令建立名為 `ood_jupyter_python` 的虛擬環境並安裝 `jupyter` 與 `jupyterlab` ```bash conda create -n ood_jupyter_python jupyter jupyterlab ``` - 建立完成後，請執行下列指令確認虛擬環境是否建立於 `/home/<username>/miniconda3/envs/ood_jupyter_python` ```bash conda env list ``` **4. 將虛擬環境註冊至 Jupyter Kernel** - 執行`conda activate ood_jupyter_python` 切換到 ood_jupyter_python 虛擬環境 ``` [user@ilgn01 ~]$ conda activate ood_jupyter_python ``` **5. 安裝其他 Python 套件** - 執行`conda list`查看目前此虛擬環境安裝了哪些套件 ``` (ood_jupyter_python) [user@ilgn01 ~]$ conda list # packages in environment at /home/user/miniconda3/envs/ood_jupyter_python: # Name Version Build Channel _libgcc_mutex 0.1 main _openmp_mutex 5.1 1_gnu anyio 4.7.0 ... ``` - 可依需求安裝所需的 Python 套件（以下為範例所需的安裝指令） ``` # 安裝深度學習套件 pip install keras tensorflow numpy matplotlib # 安裝資料處理套件 pip install numpy pandas scipy scikit-learn # 安裝視覺化套件 pip install matplotlib seaborn ``` 完成上述步驟後，所建立的虛擬環境已成功整合至 Open OnDemand 的 Jupyter Kernel 中。後續如需安裝額外套件，僅需於該虛擬環境中安裝，即可在 Jupyter 中同步使用。 <br> **6. 安裝 Jupyter Kernel 掛載於 Open OnDemand** ``` [user@ilgn01 ~]$ pip install jupyter ipykernel ``` - 註冊環境`ood_jupyter_python`到 Jupyter ``` python -m ipykernel install --user --name=ood_jupyter_python --display-name "Python (ood_jupyter_python)" ``` <br> --- <br> ## 返回 Open OnDemand 開啟 <br>Jupyter Notebook / JupyterLab 應用程式 1. 從首頁點選 **Jupyter Notebook / JupyterLab App** 圖示開啟  2. 開啟後**填寫資源配置表單**，此處使用自定義的 Miniconda Environment `ood_jupyter_python` 啟用，並點選 **「Launch」** 按鈕以執行 Jupyter 任務  3. 若 Job 成功啟動，系統將出現以下對話視窗，點選「Connect to Jupyter」來執行任務  4. 開啟 Jupyter Notebook 編輯器 - 點選 `File → New → Notebook` 來建立一個新的腳本  - 系統會彈出 **Select Kernel** 對話框，請選擇您建立的虛擬環境（例如：`Python(ood_jupyter_python)`）  - 可視需要將 Notebook 腳本重新命名，以便後續管理與辨識  - 確認既有檔案已透過傳輸檔案節點上傳  <br> --- <br> ## 程式範例執行說明 ### 1. 設置環境套件與資料載入 ``` import os import numpy as np import pandas as pd import matplotlib.pyplot as plt import tensorflow as tf from tensorflow import keras from tensorflow.keras import layers from sklearn.preprocessing import StandardScaler from sklearn.metrics import mean_absolute_error, mean_squared_error CSV_PATH = "a05000201-213521473.csv" df = pd.read_csv(CSV_PATH) print(df.head()) ``` <font color="blue">➤ 撰寫完成後，點選工具列上的符號 「▶︎」按鈕即可執行程式</font>  ### 2. 資料預處理 ``` # 改中英文欄位名稱 rename_map = { "統計期": "year", "平均氣溫[0C]": "avg_temp_c", "平均相對溼度[%]": "avg_rh", "日照時數[小時]": "sun_hours", "降水量[毫米]": "precip_mm", "降水日數[日]": "rainy_days" } # 針對 df 資料檔做改名 df.rename(columns=rename_map, inplace=True) cols = df.columns.tolist() for c in list(rename_map.keys()): if c in cols: df_raw.rename(columns={c: rename_map[c]}, inplace=True) # 檢查欄位是否存在 if "year" not in df.columns: raise ValueError("找不到『統計期/ year』欄位") FEATURES = [ col for col in ["avg_temp_c", "avg_rh", "sun_hours", "precip_mm", "rainy_days"] if col in df.columns ] # 把非數值轉成 NaN for col in FEATURES: df[col] = pd.to_numeric(df[col], errors="coerce") # 缺值補齊 df[FEATURES] = df[FEATURES].interpolate( method="linear", limit_direction="both" ) # 保留需要的欄位並去掉仍為 NaN 的列 df= df[["year"] + FEATURES].dropna().reset_index(drop=True) print(df.head()) ```  ``` # 把民國轉西元年 df["year"] = ( df["year"] .astype(str) .str.replace("年", "", regex=False) .str.strip() ) df["year"] = df["year"].astype(int) df["year"] = np.where(df["year"] >= 1911, df["year"], df["year"] + 1911) df = df.sort_values(by="year").reset_index(drop=True) print("資料筆數：", len(df)) df.head() ```  ### 3. 原始資料視覺化 ``` titles = { "avg_temp_c": "Average Temperature (°C)", "avg_rh": "Average Relative Humidity (%)", "sun_hours": "Sunshine Hours (hr)", "precip_mm": "Precipitation (mm)", "rainy_days": "Rainy Days (days)", } colors = { "avg_temp_c": "tab:red", "avg_rh": "tab:purple", "sun_hours": "tab:orange", "precip_mm": "tab:blue", "rainy_days": "tab:green", } feature_keys = ["avg_temp_c", "avg_rh", "sun_hours", "precip_mm", "rainy_days"] for k in feature_keys: if k in df.columns: df[k] = pd.to_numeric(df[k], errors="coerce") df_plot = df[["year"] + [k for k in feature_keys if k in df.columns]].dropna(subset=["year"]) df_plot = df_plot.sort_values("year").reset_index(drop=True) years = df_plot["year"].astype(int) keys_present = [k for k in feature_keys if k in df_plot.columns] n = len(keys_present) ncols = 2 nrows = (n + ncols - 1) // ncols fig, axes = plt.subplots(nrows=nrows, ncols=ncols, figsize=(14, 4 * nrows), dpi=100) axes = np.array(axes).reshape(-1) for i, k in enumerate(keys_present): ax = axes[i] ax.plot(years, df_plot[k].values, marker="o", linewidth=1, color=colors.get(k, None)) ax.set_title(titles.get(k, k)) ax.set_xlabel("Year") ax.grid(True, alpha=0.25) # 刪除空白圖 for j in range(i + 1, len(axes)): axes[j].axis("off") plt.tight_layout() plt.show() ```  ### 4. 訓練資料集（目標欄位：平均氣溫） ``` TARGET_COL = "avg_temp_c" # 切分時間比例：70% 訓練、15% 驗證、15% 測試 n = len(df) train_end = int(n * 0.7) val_end = int(n * 0.85) df_train = df.iloc[:train_end] df_val = df.iloc[train_end:val_end] df_test = df.iloc[val_end:] print(len(df_train), len(df_val), len(df_test), "（訓練/驗證/測試）") # 標準化特徵 scaler = StandardScaler() X_train = scaler.fit_transform(df_train[FEATURES].values) X_val = scaler.transform(df_val[FEATURES].values) X_test = scaler.transform(df_test[FEATURES].values) y_train = df_train[TARGET_COL].values y_val = df_val[TARGET_COL].values y_test = df_test[TARGET_COL].values ```  ### 5. 訓練時間序列 LSTM 模型 ``` class WindowSize: def __init__(self, past_years=5, horizon=1, stride=1): self.past_years = past_years self.horizon = horizon self.stride = stride def make_windows(X, y, spec: WindowSize): xs, ys = [], [] total = len(X) window = spec.past_years for start in range(0, total - window - spec.horizon + 1, spec.stride): end = start + window target_index = end + spec.horizon - 1 xs.append(X[start:end]) ys.append(y[target_index]) return np.array(xs), np.array(ys) spec = WindowSize(past_years=5, horizon=1, stride=1) Xtr_seq, ytr_seq = make_windows(X_train, y_train, spec) Xva_seq, yva_seq = make_windows(X_val, y_val, spec) Xte_seq, yte_seq = make_windows(X_test, y_test, spec) print("Xtr_seq:", Xtr_seq.shape, "ytr_seq:", ytr_seq.shape) print("Xva_seq:", Xva_seq.shape, "yva_seq:", yva_seq.shape) print("Xte_seq:", Xte_seq.shape, "yte_seq:", yte_seq.shape) ```  ``` def build_lstm(input_shape): inputs = layers.Input(shape=input_shape) x = layers.LSTM(64, return_sequences=False)(inputs) x = layers.Dropout(0.2)(x) x = layers.Dense(32, activation="relu")(x) outputs = layers.Dense(1)(x) model = keras.Model(inputs, outputs) model.compile(optimizer=keras.optimizers.Adam(1e-3), loss="mse") return model # 建立LSTM模型 model = build_lstm(Xtr_seq.shape[1:]) model.summary() callbacks = [ keras.callbacks.EarlyStopping(patience=10, restore_best_weights=True, monitor="val_loss"), keras.callbacks.ReduceLROnPlateau(patience=5, factor=0.5, min_lr=1e-5, monitor="val_loss"), ] ```  ``` # 訓練 history = model.fit( Xtr_seq, ytr_seq, validation_data=(Xva_seq, yva_seq), epochs=200, batch_size=16, callbacks=callbacks, verbose=1 ) plt.figure(figsize=(6,4)) plt.plot(history.history["loss"], label="Training loss") plt.plot(history.history["val_loss"], label="Validation loss") plt.title("Training and Validation Loss") plt.xlabel("Epoch") plt.ylabel("Loss") plt.legend() plt.show() ```  ### 6. 過往氣溫資料測試驗證測試 ``` last_k = 15 ALL_X = scaler.transform(df[FEATURES].values) ALL_y = df[TARGET_COL].values y_true, y_pred, yr = [], [], [] for i in range(len(ALL_X) - spec.past_years - spec.horizon + 1 - (last_k-1), len(ALL_X) - spec.past_years - spec.horizon + 1): start = i end = i + spec.past_years target_idx = end + spec.horizon - 1 X_win = ALL_X[start:end] # (past_years, features) y_t = ALL_y[target_idx] y_hat = model.predict(X_win[np.newaxis, :, :], verbose=0).ravel()[0] y_true.append(y_t) y_pred.append(y_hat) yr.append(years[target_idx]) y_true = np.array(y_true); y_pred = np.array(y_pred); yr = np.array(yr) plt.figure(figsize=(10,4)) plt.plot(yr, y_true, marker="o", label="Actual") plt.plot(yr, y_pred, marker="x", label="Predicted") plt.title(f"Walk-forward backtest (last {last_k} years) – {TARGET_COL}") plt.xlabel("Year"); plt.ylabel(TARGET_COL); plt.grid(True, alpha=.3); plt.legend() plt.show() mae = mean_absolute_error(y_true, y_pred) rmse = np.sqrt(mean_squared_error(y_true, y_pred)) print(f"MAE: {mae:.3f}") print(f"RMSE: {rmse:.3f}") ```  ### 7. 預測氣候資料 ``` years = df["year"].astype(int).to_numpy() past = spec.past_years if ALL_X.shape[0] < past: raise ValueError("資料年份不足，無法組成足夠 Windowsize。") hist = slice(-past, None) years_hist = years[hist] y_hist = ALL_y[hist] last_window_X = ALL_X[hist][None, ...] next_year = years[-1] + spec.horizon pred_next = float(model.predict(last_window_X, verbose=0).squeeze()) print(f"使用 {years_hist} → 預測 {next_year} 年的 {TARGET_COL} ≈ {pred_next:.2f}") plt.figure(figsize=(8, 4)) plt.plot(years_hist, y_hist, marker="o", label=f"History ({TARGET_COL})") plt.scatter([next_year], [pred_next], color="crimson", s=90, zorder=3, label=f"Forecast {next_year}: {pred_next:.2f}") plt.title(f"{TARGET_COL} – Last {past} years & Next-year Forecast") plt.xlabel("Year"); plt.ylabel(TARGET_COL) plt.grid(True, alpha=.3); plt.legend() plt.show() ```  <br> <br> <br> <!-- 作者資訊 --> <div style="border-top: 2px solid #eee; padding: 28px 20px; display: flex; justify-content: space-between; align-items: center; font-family: sans-serif;"> <div> <div style="font-size: 20px; font-weight: bold; margin-bottom: 8px;"> Jupyter Lab 操作範例說明</div> <div style="max-width: 640px; color: #444; line-height: ;"> Author：Hsin-Jou Chan <br>Date：2025-08-15</div> </div> <div> <a href="https://man.twcc.ai/@f1-manual/manual" target="_blank" style="display: inline-block;"> <img src=" https://iservice.nchc.org.tw/nchc_service/images/nchc/pi4.png" alt="創進一號使用說明" style="height: 80px;"> </a> </div> </div>