rltrader
rltrader copied to clipboard
quantylab
마지막 epoch에서는 매매를 하지 않네요. 버그인가요?
아래와 같이 epoch 후반으로 갈 수록 매매횟수가 급격히 줄어들더니 마지막 epoch에서는 아예 매매를 하지 않습니다.
올려주신 예제를 그대로 실행한 결과입니다. A2C로 하건 A3C로 하건 마찬가지네요. 어디서 잘못된 것일까요?
{"mode": "train", "ver": "v3", "name": "005930", "stock_code": ["005930"], "rl_method": "a3c", "net": "lstm", "backend": "pytorch", "start_date": "20180101", "end_date": "20191231", "lr": 0.0001, "discount_factor": 0.7, "balance": 100000000} [005930] RL:a3c NET:lstm LR:0.0001 DF:0.7 [005930][Epoch 001/100] Epsilon:1.0000 #Expl.:486/486 #Buy:159 #Sell:135 #Hold:192 #Stocks:136 PV:90,758,965 Loss:0.709582 ET:2.9346 [005930][Epoch 002/100] Epsilon:0.9899 #Expl.:478/486 #Buy:152 #Sell:133 #Hold:201 #Stocks:699 PV:105,518,703 Loss:0.714348 ET:2.6699 [005930][Epoch 003/100] Epsilon:0.9798 #Expl.:479/486 #Buy:169 #Sell:141 #Hold:176 #Stocks:1170 PV:124,760,318 Loss:0.738551 ET:2.7686 [005930][Epoch 004/100] Epsilon:0.9697 #Expl.:464/486 #Buy:149 #Sell:137 #Hold:200 #Stocks:83 PV:105,708,056 Loss:0.710187 ET:2.6629 [005930][Epoch 005/100] Epsilon:0.9596 #Expl.:458/486 #Buy:167 #Sell:147 #Hold:172 #Stocks:486 PV:95,198,980 Loss:0.704760 ET:2.7715 [005930][Epoch 006/100] Epsilon:0.9495 #Expl.:457/486 #Buy:141 #Sell:130 #Hold:215 #Stocks:0 PV:95,734,055 Loss:0.702496 ET:2.7536 [005930][Epoch 007/100] Epsilon:0.9394 #Expl.:459/486 #Buy:162 #Sell:145 #Hold:179 #Stocks:179 PV:101,612,578 Loss:0.706646 ET:2.6818 [005930][Epoch 008/100] Epsilon:0.9293 #Expl.:443/486 #Buy:155 #Sell:144 #Hold:187 #Stocks:84 PV:97,803,346 Loss:0.704463 ET:2.7387 [005930][Epoch 009/100] Epsilon:0.9192 #Expl.:441/486 #Buy:151 #Sell:141 #Hold:194 #Stocks:0 PV:96,104,865 Loss:0.702421 ET:2.7535 [005930][Epoch 010/100] Epsilon:0.9091 #Expl.:447/486 #Buy:154 #Sell:136 #Hold:196 #Stocks:336 PV:106,843,758 Loss:0.711046 ET:2.7347 [005930][Epoch 011/100] Epsilon:0.8990 #Expl.:439/486 #Buy:144 #Sell:138 #Hold:204 #Stocks:0 PV:101,983,692 Loss:0.705364 ET:2.7068 [005930][Epoch 012/100] Epsilon:0.8889 #Expl.:425/486 #Buy:142 #Sell:135 #Hold:209 #Stocks:0 PV:98,495,887 Loss:0.703558 ET:2.6763 [005930][Epoch 013/100] Epsilon:0.8788 #Expl.:427/486 #Buy:134 #Sell:129 #Hold:223 #Stocks:0 PV:101,575,168 Loss:0.704008 ET:2.7656 [005930][Epoch 014/100] Epsilon:0.8687 #Expl.:416/486 #Buy:135 #Sell:126 #Hold:225 #Stocks:366 PV:103,799,870 Loss:0.705671 ET:2.6828 [005930][Epoch 015/100] Epsilon:0.8586 #Expl.:410/486 #Buy:142 #Sell:138 #Hold:206 #Stocks:0 PV:102,817,781 Loss:0.703574 ET:2.7028 [005930][Epoch 016/100] Epsilon:0.8485 #Expl.:385/486 #Buy:116 #Sell:113 #Hold:257 #Stocks:0 PV:99,107,645 Loss:0.701767 ET:2.7833 [005930][Epoch 017/100] Epsilon:0.8384 #Expl.:408/486 #Buy:130 #Sell:116 #Hold:240 #Stocks:739 PV:104,026,015 Loss:0.705632 ET:2.6970 [005930][Epoch 018/100] Epsilon:0.8283 #Expl.:403/486 #Buy:134 #Sell:125 #Hold:227 #Stocks:253 PV:101,834,537 Loss:0.703947 ET:2.7573 [005930][Epoch 019/100] Epsilon:0.8182 #Expl.:379/486 #Buy:133 #Sell:130 #Hold:223 #Stocks:0 PV:96,518,255 Loss:0.701386 ET:2.8055 [005930][Epoch 020/100] Epsilon:0.8081 #Expl.:401/486 #Buy:130 #Sell:126 #Hold:230 #Stocks:0 PV:102,011,389 Loss:0.702306 ET:2.7415 [005930][Epoch 021/100] Epsilon:0.7980 #Expl.:389/486 #Buy:131 #Sell:128 #Hold:227 #Stocks:0 PV:102,119,181 Loss:0.703019 ET:2.6761 [005930][Epoch 022/100] Epsilon:0.7879 #Expl.:372/486 #Buy:135 #Sell:132 #Hold:219 #Stocks:0 PV:100,957,517 Loss:0.702800 ET:2.6918 [005930][Epoch 023/100] Epsilon:0.7778 #Expl.:377/486 #Buy:116 #Sell:115 #Hold:255 #Stocks:0 PV:95,384,828 Loss:0.700411 ET:2.7533 [005930][Epoch 024/100] Epsilon:0.7677 #Expl.:375/486 #Buy:129 #Sell:127 #Hold:230 #Stocks:0 PV:98,256,387 Loss:0.700826 ET:2.6973 [005930][Epoch 025/100] Epsilon:0.7576 #Expl.:357/486 #Buy:108 #Sell:107 #Hold:271 #Stocks:0 PV:102,070,065 Loss:0.700553 ET:2.7487 [005930][Epoch 026/100] Epsilon:0.7475 #Expl.:358/486 #Buy:131 #Sell:129 #Hold:226 #Stocks:0 PV:99,856,870 Loss:0.701592 ET:2.7026 [005930][Epoch 027/100] Epsilon:0.7374 #Expl.:355/486 #Buy:128 #Sell:123 #Hold:235 #Stocks:238 PV:96,109,249 Loss:0.700117 ET:2.6965 [005930][Epoch 028/100] Epsilon:0.7273 #Expl.:356/486 #Buy:118 #Sell:115 #Hold:253 #Stocks:0 PV:98,956,027 Loss:0.701214 ET:2.7122 [005930][Epoch 029/100] Epsilon:0.7172 #Expl.:352/486 #Buy:104 #Sell:103 #Hold:279 #Stocks:84 PV:99,460,696 Loss:0.699762 ET:2.7093 [005930][Epoch 030/100] Epsilon:0.7071 #Expl.:352/486 #Buy:114 #Sell:111 #Hold:261 #Stocks:0 PV:98,382,992 Loss:0.700129 ET:2.6713 [005930][Epoch 031/100] Epsilon:0.6970 #Expl.:345/486 #Buy:121 #Sell:120 #Hold:245 #Stocks:0 PV:97,963,739 Loss:0.700655 ET:2.6127 [005930][Epoch 032/100] Epsilon:0.6869 #Expl.:327/486 #Buy:108 #Sell:105 #Hold:273 #Stocks:169 PV:98,453,449 Loss:0.699436 ET:2.7340 [005930][Epoch 033/100] Epsilon:0.6768 #Expl.:325/486 #Buy:104 #Sell:103 #Hold:279 #Stocks:0 PV:99,084,560 Loss:0.699247 ET:2.7699 [005930][Epoch 034/100] Epsilon:0.6667 #Expl.:322/486 #Buy:113 #Sell:109 #Hold:264 #Stocks:252 PV:98,972,798 Loss:0.699663 ET:2.8325 [005930][Epoch 035/100] Epsilon:0.6566 #Expl.:326/486 #Buy:109 #Sell:105 #Hold:272 #Stocks:84 PV:97,388,976 Loss:0.699912 ET:2.7250 [005930][Epoch 036/100] Epsilon:0.6465 #Expl.:315/486 #Buy:104 #Sell:102 #Hold:280 #Stocks:0 PV:97,519,808 Loss:0.699118 ET:2.8010 [005930][Epoch 037/100] Epsilon:0.6364 #Expl.:306/486 #Buy:107 #Sell:106 #Hold:273 #Stocks:0 PV:98,985,992 Loss:0.699206 ET:2.9005 [005930][Epoch 038/100] Epsilon:0.6263 #Expl.:302/486 #Buy:98 #Sell:96 #Hold:292 #Stocks:0 PV:99,529,562 Loss:0.698646 ET:2.7822 [005930][Epoch 039/100] Epsilon:0.6162 #Expl.:290/486 #Buy:89 #Sell:88 #Hold:309 #Stocks:71 PV:99,509,260 Loss:0.698614 ET:2.7428 [005930][Epoch 040/100] Epsilon:0.6061 #Expl.:298/486 #Buy:103 #Sell:100 #Hold:283 #Stocks:225 PV:97,365,616 Loss:0.698516 ET:2.7255 [005930][Epoch 041/100] Epsilon:0.5960 #Expl.:284/486 #Buy:101 #Sell:99 #Hold:286 #Stocks:85 PV:98,323,451 Loss:0.698460 ET:2.6558 [005930][Epoch 042/100] Epsilon:0.5859 #Expl.:291/486 #Buy:93 #Sell:93 #Hold:300 #Stocks:0 PV:97,043,880 Loss:0.698096 ET:2.7185 [005930][Epoch 043/100] Epsilon:0.5758 #Expl.:291/486 #Buy:96 #Sell:95 #Hold:295 #Stocks:0 PV:100,169,291 Loss:0.698939 ET:2.7614 [005930][Epoch 044/100] Epsilon:0.5657 #Expl.:275/486 #Buy:96 #Sell:95 #Hold:295 #Stocks:85 PV:100,512,686 Loss:0.698530 ET:2.7402 [005930][Epoch 045/100] Epsilon:0.5556 #Expl.:260/486 #Buy:90 #Sell:90 #Hold:306 #Stocks:0 PV:99,434,000 Loss:0.697760 ET:2.8257 [005930][Epoch 046/100] Epsilon:0.5455 #Expl.:270/486 #Buy:91 #Sell:91 #Hold:304 #Stocks:0 PV:99,396,466 Loss:0.698101 ET:2.6829 [005930][Epoch 047/100] Epsilon:0.5354 #Expl.:264/486 #Buy:86 #Sell:85 #Hold:315 #Stocks:0 PV:101,525,713 Loss:0.698313 ET:2.8206 [005930][Epoch 048/100] Epsilon:0.5253 #Expl.:281/486 #Buy:82 #Sell:78 #Hold:326 #Stocks:339 PV:99,631,395 Loss:0.697686 ET:2.7653 [005930][Epoch 049/100] Epsilon:0.5152 #Expl.:238/486 #Buy:75 #Sell:73 #Hold:338 #Stocks:169 PV:99,276,517 Loss:0.697204 ET:2.6898 [005930][Epoch 050/100] Epsilon:0.5051 #Expl.:267/486 #Buy:90 #Sell:90 #Hold:306 #Stocks:0 PV:99,491,395 Loss:0.698139 ET:2.6416 [005930][Epoch 051/100] Epsilon:0.4949 #Expl.:245/486 #Buy:83 #Sell:83 #Hold:320 #Stocks:0 PV:98,502,636 Loss:0.697194 ET:2.7121 [005930][Epoch 052/100] Epsilon:0.4848 #Expl.:232/486 #Buy:75 #Sell:74 #Hold:337 #Stocks:85 PV:100,903,852 Loss:0.697387 ET:2.7443 [005930][Epoch 053/100] Epsilon:0.4747 #Expl.:246/486 #Buy:79 #Sell:79 #Hold:328 #Stocks:0 PV:99,125,404 Loss:0.696994 ET:2.6373 [005930][Epoch 054/100] Epsilon:0.4646 #Expl.:245/486 #Buy:79 #Sell:78 #Hold:329 #Stocks:85 PV:100,150,340 Loss:0.697160 ET:2.7274 [005930][Epoch 055/100] Epsilon:0.4545 #Expl.:208/486 #Buy:63 #Sell:63 #Hold:360 #Stocks:0 PV:101,002,033 Loss:0.696571 ET:2.8155 [005930][Epoch 056/100] Epsilon:0.4444 #Expl.:205/486 #Buy:73 #Sell:73 #Hold:340 #Stocks:0 PV:98,356,064 Loss:0.696904 ET:2.7664 [005930][Epoch 057/100] Epsilon:0.4343 #Expl.:216/486 #Buy:77 #Sell:77 #Hold:332 #Stocks:0 PV:98,138,133 Loss:0.696758 ET:2.7058 [005930][Epoch 058/100] Epsilon:0.4242 #Expl.:214/486 #Buy:64 #Sell:64 #Hold:358 #Stocks:0 PV:98,417,130 Loss:0.696335 ET:2.7948 [005930][Epoch 059/100] Epsilon:0.4141 #Expl.:203/486 #Buy:73 #Sell:73 #Hold:340 #Stocks:0 PV:99,685,268 Loss:0.696678 ET:2.6479 [005930][Epoch 060/100] Epsilon:0.4040 #Expl.:213/486 #Buy:67 #Sell:66 #Hold:353 #Stocks:85 PV:100,651,063 Loss:0.696705 ET:2.7623 [005930][Epoch 061/100] Epsilon:0.3939 #Expl.:196/486 #Buy:47 #Sell:46 #Hold:393 #Stocks:85 PV:100,160,685 Loss:0.695683 ET:2.6995 [005930][Epoch 062/100] Epsilon:0.3838 #Expl.:200/486 #Buy:68 #Sell:67 #Hold:351 #Stocks:0 PV:99,719,398 Loss:0.696632 ET:2.7527 [005930][Epoch 063/100] Epsilon:0.3737 #Expl.:179/486 #Buy:66 #Sell:66 #Hold:354 #Stocks:0 PV:100,003,164 Loss:0.696085 ET:2.6862 [005930][Epoch 064/100] Epsilon:0.3636 #Expl.:185/486 #Buy:62 #Sell:62 #Hold:362 #Stocks:0 PV:99,542,394 Loss:0.696198 ET:2.7161 [005930][Epoch 065/100] Epsilon:0.3535 #Expl.:174/486 #Buy:54 #Sell:54 #Hold:378 #Stocks:0 PV:99,093,886 Loss:0.695744 ET:2.8497 [005930][Epoch 066/100] Epsilon:0.3434 #Expl.:173/486 #Buy:59 #Sell:59 #Hold:368 #Stocks:0 PV:99,128,984 Loss:0.695933 ET:3.0130 [005930][Epoch 067/100] Epsilon:0.3333 #Expl.:148/486 #Buy:47 #Sell:47 #Hold:392 #Stocks:0 PV:99,550,237 Loss:0.695236 ET:3.1447 [005930][Epoch 068/100] Epsilon:0.3232 #Expl.:134/486 #Buy:43 #Sell:43 #Hold:400 #Stocks:0 PV:99,979,142 Loss:0.695095 ET:3.0293 [005930][Epoch 069/100] Epsilon:0.3131 #Expl.:144/486 #Buy:61 #Sell:61 #Hold:364 #Stocks:0 PV:99,125,478 Loss:0.695587 ET:3.2653 [005930][Epoch 070/100] Epsilon:0.3030 #Expl.:146/486 #Buy:46 #Sell:44 #Hold:396 #Stocks:169 PV:99,479,908 Loss:0.695188 ET:2.7563 [005930][Epoch 071/100] Epsilon:0.2929 #Expl.:160/486 #Buy:58 #Sell:58 #Hold:370 #Stocks:0 PV:98,531,407 Loss:0.695663 ET:2.7207 [005930][Epoch 072/100] Epsilon:0.2828 #Expl.:126/486 #Buy:42 #Sell:42 #Hold:402 #Stocks:0 PV:99,665,999 Loss:0.694862 ET:2.7466 [005930][Epoch 073/100] Epsilon:0.2727 #Expl.:146/486 #Buy:54 #Sell:53 #Hold:379 #Stocks:85 PV:98,924,021 Loss:0.695341 ET:2.7040 [005930][Epoch 074/100] Epsilon:0.2626 #Expl.:134/486 #Buy:45 #Sell:45 #Hold:396 #Stocks:0 PV:98,828,775 Loss:0.694999 ET:2.7353 [005930][Epoch 075/100] Epsilon:0.2525 #Expl.:115/486 #Buy:38 #Sell:38 #Hold:410 #Stocks:0 PV:99,479,114 Loss:0.694702 ET:2.6150 [005930][Epoch 076/100] Epsilon:0.2424 #Expl.:117/486 #Buy:45 #Sell:45 #Hold:396 #Stocks:0 PV:99,733,521 Loss:0.694773 ET:2.7008 [005930][Epoch 077/100] Epsilon:0.2323 #Expl.:122/486 #Buy:39 #Sell:39 #Hold:408 #Stocks:0 PV:99,698,149 Loss:0.694734 ET:2.6726 [005930][Epoch 078/100] Epsilon:0.2222 #Expl.:123/486 #Buy:32 #Sell:32 #Hold:422 #Stocks:0 PV:99,794,743 Loss:0.694435 ET:2.7716 [005930][Epoch 079/100] Epsilon:0.2121 #Expl.:102/486 #Buy:30 #Sell:30 #Hold:426 #Stocks:0 PV:99,669,297 Loss:0.694401 ET:2.6676 [005930][Epoch 080/100] Epsilon:0.2020 #Expl.:128/486 #Buy:45 #Sell:45 #Hold:396 #Stocks:0 PV:99,687,310 Loss:0.694976 ET:2.6968 [005930][Epoch 081/100] Epsilon:0.1919 #Expl.:97/486 #Buy:44 #Sell:44 #Hold:398 #Stocks:0 PV:99,311,177 Loss:0.694767 ET:2.7218 [005930][Epoch 082/100] Epsilon:0.1818 #Expl.:88/486 #Buy:43 #Sell:43 #Hold:400 #Stocks:0 PV:99,899,386 Loss:0.694638 ET:2.7016 [005930][Epoch 083/100] Epsilon:0.1717 #Expl.:82/486 #Buy:42 #Sell:42 #Hold:402 #Stocks:0 PV:100,379,349 Loss:0.694301 ET:2.8211 [005930][Epoch 084/100] Epsilon:0.1616 #Expl.:72/486 #Buy:25 #Sell:25 #Hold:436 #Stocks:0 PV:99,709,487 Loss:0.693879 ET:2.8118 [005930][Epoch 085/100] Epsilon:0.1515 #Expl.:75/486 #Buy:24 #Sell:24 #Hold:438 #Stocks:0 PV:99,819,234 Loss:0.693761 ET:2.7095 [005930][Epoch 086/100] Epsilon:0.1414 #Expl.:67/486 #Buy:22 #Sell:22 #Hold:442 #Stocks:0 PV:99,622,580 Loss:0.693790 ET:2.7252 [005930][Epoch 087/100] Epsilon:0.1313 #Expl.:78/486 #Buy:27 #Sell:27 #Hold:432 #Stocks:0 PV:100,700,567 Loss:0.693821 ET:2.6877 [005930][Epoch 088/100] Epsilon:0.1212 #Expl.:64/486 #Buy:16 #Sell:16 #Hold:454 #Stocks:0 PV:100,083,247 Loss:0.693422 ET:2.9005 [005930][Epoch 089/100] Epsilon:0.1111 #Expl.:44/486 #Buy:18 #Sell:18 #Hold:450 #Stocks:0 PV:100,267,027 Loss:0.693419 ET:2.7417 [005930][Epoch 090/100] Epsilon:0.1010 #Expl.:53/486 #Buy:21 #Sell:21 #Hold:444 #Stocks:0 PV:99,798,020 Loss:0.693556 ET:2.7909 [005930][Epoch 091/100] Epsilon:0.0909 #Expl.:52/486 #Buy:19 #Sell:19 #Hold:448 #Stocks:0 PV:99,473,125 Loss:0.693487 ET:2.7105 [005930][Epoch 092/100] Epsilon:0.0808 #Expl.:48/486 #Buy:13 #Sell:13 #Hold:460 #Stocks:0 PV:100,092,871 Loss:0.693190 ET:2.7746 [005930][Epoch 093/100] Epsilon:0.0707 #Expl.:32/486 #Buy:17 #Sell:17 #Hold:452 #Stocks:0 PV:99,963,554 Loss:0.693162 ET:2.6559 [005930][Epoch 094/100] Epsilon:0.0606 #Expl.:31/486 #Buy:15 #Sell:15 #Hold:456 #Stocks:0 PV:99,412,186 Loss:0.693221 ET:2.7804 [005930][Epoch 095/100] Epsilon:0.0505 #Expl.:18/486 #Buy:5 #Sell:5 #Hold:476 #Stocks:0 PV:99,567,275 Loss:0.692919 ET:2.6876 [005930][Epoch 096/100] Epsilon:0.0404 #Expl.:23/486 #Buy:6 #Sell:6 #Hold:474 #Stocks:0 PV:99,875,608 Loss:0.692862 ET:2.6774 [005930][Epoch 097/100] Epsilon:0.0303 #Expl.:14/486 #Buy:5 #Sell:5 #Hold:476 #Stocks:0 PV:99,899,800 Loss:0.692807 ET:2.7310 [005930][Epoch 098/100] Epsilon:0.0202 #Expl.:17/486 #Buy:4 #Sell:4 #Hold:478 #Stocks:0 PV:99,920,417 Loss:0.692798 ET:2.6739 [005930][Epoch 099/100] Epsilon:0.0101 #Expl.:9/486 #Buy:2 #Sell:2 #Hold:482 #Stocks:0 PV:100,067,176 Loss:0.692623 ET:2.7746 [005930][Epoch 100/100] Epsilon:0.0000 #Expl.:0/486 #Buy:0 #Sell:0 #Hold:486 #Stocks:0 PV:100,000,000 Loss:0.692483 ET:2.7496 [005930] Elapsed Time:327.5504 Max PV:124,760,318 #Win:30
매수 가치를 제대로 학습하지 못한 경우로 보입니다. 다음의 시도가 도움이 될 수 있겠습니다.
- 매매 비용(세금 + 수수료) 없이 학습하기
- Learning Rate 줄이고 Epoch 늘리기
- 탐험률 높이기
