--- title: 資料轉移T3->F1 | zh tags: Guide, TWNIA3, TW GA: --- {%hackmd @docsharedstyle/default %} <div style="text-align: right">更新日期:2024/06/26</div> 台灣杉三號資料轉移至創進一號說明 === <!-- ![downloads](https://img.shields.io/github/downloads/atom/atom/total.svg) ![build](https://img.shields.io/appveyor/ci/:user/:repo.svg) ![chat](https://img.shields.io/discord/:serverId.svg) --> [TOC] ## 登入資訊 <table> <tr> <th>類別</th> <th>創進一號</th> <th>台灣杉三號</th> </tr> <tr> <th>登入節點</th> <th>f1-ilgn01.nchc.org.tw<br>f1-ilgn02.nchc.org.tw</th> <th>twnia3.nchc.org.tw<br>twnia3a.nchc.org.tw</th> </tr> <tr> <th>傳輸節點</th> <th>f1-dtn01.nchc.org.tw<br>f1-dtn02.nchc.org.tw</th> <th>t3-x1.nchc.org.tw<br>t3-x2.nchc.org.tw</th> </tr> <tr> <th>家目錄</th> <th>/home/<i>$user</i><sup>1</sup></th> <th>/home/<i>$user</i></th> </tr> <tr> <th>工作目錄</th> <th>/work1/<i>$user</i></th> <th>/work/<i>$user</i></th> </tr> <tr> <th>Project目錄</th> <th>/project/<i>$計畫代號</i><sup>2</sup></th> <th>/project/<i>$計畫代號</i></th> </tr> </table> :::info <i class="fa fa-bullhorn" aria-hidden="true"></i> **備註**: 1.<b><i>&#36;user</i></b>為使用者帳號,勿直接使用<i>$user</i> 2.<b><i>&#36;計畫代號</i></b>為使用者於國網中心使用之計畫編號,如MST113001 ::: --- :::success **由台灣杉三號至創進一號傳輸檔案資料** ::: 製作與傳送SSH金鑰 --- 1. 以帳號、密碼、OTP登入台灣杉三號登入節點 ``` login as: demo_user Keyboard-interactive authentication prompts from server: | Password: | Changing MOTP: 537268 End of keyboard-interactive prompts from server Auth MOTP: PASS Auth Password: PASS Last login: Tue Dec 26 15:26:33 2023 ... [demo_user@lgn301 ~]$ ``` 2. 以ssh-keygen製作SSH金鑰 [demo_user@lgn301 ~]$ ssh-keygen <b>-t rsa</b> > rsa可替換為dsa、ecdsa、ed25519、rsa1或使用預設值(不加任何參數)亦可 ``` Generating public/private rsa key pair. Enter file in which to save the key (/home/demo_user/.ssh/id_rsa): Enter passphrase (empty for no passphrase): <--建議輸入密碼保護,省略亦可使用,但有安全性疑慮 Enter same passphrase again: Your identification has been saved in /home/demo_user/.ssh/id_rsa. Your public key has been saved in /home/demo_user/.ssh/id_rsa.pub. The key fingerprint is: SHA256:RPWBSlXmUJ9ysUMxcxnqo5cj7zdkAK+qHJie1XxUovw demo_user@lgn301.nchc-2020-hpc The key's randomart image is: +---[RSA 2048]----+ | .o++= *o+| | .. .* +.O | | ... .*o* | | .o . += . | | So ..o. | | o o o.. oo | | o o o.E +o | | . + ... + .o | | o o. .o. .| +----[SHA256]-----+ ``` :::info <i class="fa fa-bullhorn" aria-hidden="true"></i> **備註**: 產生/home/demo_user/.ssh/id_rsa(私鑰) 與 /home/demo_user/.ssh/id_rsa.pub(公鑰) ::: 3. 以ssh-copy-id傳送SSH金鑰至遠端(創進一號) [demo_user@lgn301 ~]$ ssh-copy-id <b>-i /home/demo_user/.ssh/id_rsa.pub</b> demo_user@f1-ilgn01.nchc.org.tw :::info <i class="fa fa-bullhorn" aria-hidden="true"></i> **備註**: -i 指定公鑰,若非rsa型式,請依實際名稱輸入 ::: ``` /bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/home/demo_user/.ssh/id_rsa.pub" /bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed /bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys Please select the 2FA login method. 1. Mobile APP OTP 2. Mobile APP PUSH 3. Email OTP Login method: 2 Password: Number of key(s) added: 1 Now try logging into the machine, with: "ssh 'f1-ilgn01.nchc.org.tw'" and check to make sure that only the key(s) you wanted were added. [demo_user@lgn301 ~]$ ``` <!-- 傳輸方式 --- ### SCP * 傳輸檔案 1. 語法: scp 來源檔案 目的檔名/目錄 scp -r 來源目錄/檔案 目的目錄/檔案 2. 範例: [demo_user@lgn301 demo_user]$ scp demo_user@f1-dtn01.nchc.org.tw:/work1/demo_user/CentOS-7-x86_64-Everything-2009.iso /work/demo_user/ ``` Enter passphrase for key '/home/demo_user/.ssh/id_rsa': CentOS-7-x86_64-Everything-2009.iso 100% 9728MB 190.0MB/s 00:51 [demo_user@lgn301 demo_user]$ ls -l CentOS-7-x86_64-Everything-2009.iso -rw-r--r-- 1 demo_user TRI107056 10200547328 Dec 21 15:15 CentOS-7-x86_64-Everything-2009.iso ``` :::info <i class="fa fa-bullhorn" aria-hidden="true"></i> **備註**: 因二系統間帳號是一樣的,故亦可省略帳號,如:scp f1-dtn01.nchc.org.tw:/work1/demo_user/CentOS-7-x86_64-Everything-2009.iso /work/demo_user/ ::: [demo_user@lgn301 ~]$ scp -r f1-dtn01.nchc.org.tw:/work1/demo_user/*.c /work/demo_user/ ``` Enter passphrase for key '/home/demo_user/.ssh/id_rsa': cpi.c 100% 1685 88.9KB/s 00:00 hello.c 100% 123 16.5KB/s 00:00 [demo_user@lgn301 ~]$ ``` :::info <i class="fa fa-bullhorn" aria-hidden="true"></i> **備註**: 檔案名稱可使用regular expression方式表示,如*、?、.之類的字元 ::: * 傳輸目錄 1. 語法: scp -r 來源目錄 目的目錄 2. 範例: [demo_user@lgn301 demo_user]$ scp -r demo_user@f1-dtn01.nchc.org.tw:/work1/demo_user/batch /work/demo_user/ ``` Enter passphrase for key '/home/demo_user/.ssh/id_rsa': sub_ansys182fluent 100% 2192 277.6KB/s 00:00 sub_ansys190fluent 100% 2223 127.1KB/s 00:00 [demo_user@lgn301 demo_user]$ ``` --> ### SFTP * 範例 <font color=red><b>(以下為示範範例,請您依實際需求修改,勿照抄。)</b></font> [demo_user@lgn303 ~]$ sftp demo_user@f1-dtn01.nchc.org.tw ``` Enter passphrase for key '/home/demo_user/.ssh/id_rsa': <--若有密碼保護,請輸入密碼 Connected to f1-dtn01.nchc.org.tw. sftp> pwd <--顯示遠端目錄位置 Remote working directory: /home/demo_user sftp> cd /work1/demo_user <-- 切換遠端目錄至 /work1/demo_user sftp> pwd Remote working directory: /work1/demo_user sftp> lcd scripts <-- 切換local端目錄至scripts (這是範例,勿照抄) sftp> lls -l <-- 查看local的awk目錄檔案 total 1 drwxr-xr-x 2 demo_user TRI107056 4096 Oct 29 2021 awk sftp> put -r awk <-- 遞迴傳送local 的awk目錄檔案到遠端 Uploading awk/ to /work1/demo_user/awk Entering awk/ awk/jobs.awk 100% 1186 199.5KB/s 00:00 awk/jobs.awk.zjw-org 100% 1560 261.8KB/s 00:00 awk/node.awk 100% 1499 258.3KB/s 00:00 sftp> sftp> help <-- 指令功能說明 Available commands: bye Quit sftp cd path Change remote directory to 'path' chgrp grp path Change group of file 'path' to 'grp' chmod mode path Change permissions of file 'path' to 'mode' chown own path Change owner of file 'path' to 'own' df [-hi] [path] Display statistics for current directory or filesystem containing 'path' exit Quit sftp get [-afPpRr] remote [local] Download file reget [-fPpRr] remote [local] Resume download file reput [-fPpRr] [local] remote Resume upload file help Display this help text lcd path Change local directory to 'path' lls [ls-options [path]] Display local directory listing lmkdir path Create local directory ln [-s] oldpath newpath Link remote file (-s for symlink) lpwd Print local working directory ls [-1afhlnrSt] [path] Display remote directory listing lumask umask Set local umask to 'umask' mkdir path Create remote directory progress Toggle display of progress meter put [-afPpRr] local [remote] Upload file pwd Display remote working directory quit Quit sftp rename oldpath newpath Rename remote file rm path Delete remote file rmdir path Remove remote directory symlink oldpath newpath Symlink remote file version Show SFTP version !command Execute 'command' in local shell ! Escape to local shell ? Synonym for help ``` <!-- ### RSYNC 1. 語法:rsync 參數 來源檔案/目錄 目的檔案/目錄 2. 基本參數說明: -v:verbose模式,輸出較詳細的資訊 -a:封裝模式備份,等同使用參數-rlptgoD -r:遞迴(--recursive),備份目錄下的所有檔案與子目錄 -z:在傳輸過程中壓縮資料,以節省頻寬 -h:以易讀的數字(單位)顯示 --progress:傳輸過程中顯示進度 詳細參數說明,請登入系統man rsync。 範例: * 單一檔案 [demo_user@lgn303 ~]$ rsync -azvh demo_user@f1-dtn01.nchc.org.tw:/work1/demo_user/CentOS-7-x86_64-Everything-2009.iso ./ ``` Enter passphrase for key '/home/demo_user/.ssh/id_rsa': receiving incremental file list CentOS-7-x86_64-Everything-2009.iso sent 30 bytes received 10.20G bytes 131.66M bytes/sec total size is 10.20G speedup is 1.00 ``` * 顯示進度 --progress [demo_user@lgn303 ~]$ rsync -azvh --progress demo_user@f1-dtn01.nchc.org.tw:/work1/demo_user/CentOS-7-x86_64-Everything-2009.iso ./ ``` Enter passphrase for key '/home/demo_user/.ssh/id_rsa': receiving incremental file list CentOS-7-x86_64-Everything-2009.iso 1.44G 14% 138.40MB/s 0:01:01 ``` * 傳輸目錄 [demo_user@lgn303 ~]$ rsync -azvh --progress demo_user@f1-dtn01.nchc.org.tw:/work1/demo_user/hfss ./ ``` Enter passphrase for key '/home/demo_user/.ssh/id_rsa': receiving incremental file list hfss/ hfss/HFSS_test.o5328178 34.22K 100% 32.63MB/s 0:00:00 (xfr#1, to-chk=41/43) hfss/HFSS_test.o5328760 34.62K 100% 1.65MB/s 0:00:00 (xfr#2, to-chk=40/43) hfss/HFSS_test.o5328900 960 100% 44.64kB/s 0:00:00 (xfr#3, to-chk=39/43) hfss/HFSS_test.o5328919 960 100% 18.38kB/s 0:00:00 (xfr#4, to-chk=38/43) ...(略) 258 100% 0.56kB/s 0:00:00 (xfr#33, to-chk=1/43) hfss/phased_array_unitcell.aedtresults/mf_2/svcache/ hfss/phased_array_unitcell.aedtresults/mf_2/svcache/HDR4MlOn19068615771756414.tmp 320 100% 0.68kB/s 0:00:00 (xfr#34, to-chk=0/43) sent 693 bytes received 112.46K bytes 17.41K bytes/sec total size is 558.48K speedup is 4.94 ``` -->