YonV1943 曾伊言

https://github.com/AI4Finance-Foundation/ElegantRL/blob/3721f6e677ea5a2720be23bbb18855a2f06767d7/elegantrl/train/run.py#L99 In order to be compatible with the WinOS system, it needs to make this change: ``` method = 'spawn' if os.name == 'nt' else 'forkserver' # os.name == 'nt'...

I add the unit tests for `elegantrl.train.config` and `elegantrl.agents.net`. And I add `agents.net.QNetBase` to keep a same type with `ActorBase` and `CriticBase` for normalization.

The better way to get the running stat of std: ``` def update_avg_std_for_state_value_norm(self, states: Tensor, returns: Tensor): tau = self.state_value_tau state_avg = states.mean(dim=0, keepdim=True) state_vam = (states ** 2).mean(dim=0, keepdim=True)...

The [subprocess Vectorized environment of stable baselines 3](https://stable-baselines3.readthedocs.io/en/master/guide/vec_envs.html) is practically useful. So I added a simple **subprocess Vectorized environment** into ElegantRL. Demo code: Add the `num_envs=int` and `if_build_vec_env=True` to the...

fig the bug for vectorized environment. - AgentBase.__init__() - AgentXXX.explore_env() - AgentXXX.update_net() - build_mlp() - ReplayBuffer.update() This Pull Request fix the bugs that related to follosing issues: - https://github.com/AI4Finance-Foundation/ElegantRL/issues/246 -...

Fix the AgentBase.__init__ () for all the DRL algorithms in folder /elegantrl/agents

1

In the commit we still have self.act = act_class(net_dim, state_dim, action_dim).to(self.device) self.cri = cri_class(net_dim, state_dim, action_dim).to(self.device) \ if cri_class else self.act the example still crashes for me _Originally posted by...

✨ 😅 Is possibale to use the ChatGPT of OpenAI to train this ChatGPT?

8

OpenAI used **40 people** when training their own chatGPT, and the annotation process lasted for **3 months**. It is difficult for our open source community （github） to reproduce the **Reinforcement...

📝 Graph: Learning Combinatorial Optimization Algorithms over Graphs

2

我们图上的组合优化问题，例如 Graph maxcut，下面的论文提出了对图结构进行了编码的方案： - 论文 Learning Combinatorial Optimization Algorithms over Graphs （宋乐教授） - 代码 https://github.com/Hanjun-Dai/graphnn 直接复现上面的代码，因为版本问题，不容易，我有以下建议： - 建议复现他们的 PyTorch版本 - 建议安装PyTorch的 0.8.5 版本 - 建议安装 rdkit 和 boost 的最新版本 复现过程碰到问题，随时在这个issue...

📝 MCPG: special sampling methds and special local search

1

我对MCPG算法的代码又进行了简化，简化后只有 300行内。有两个地方我看不懂，希望大家讨论后一起弄懂。 1. 我想要知道MCPG代码里这个采样方法对应论文的哪里？论文是否提及了？[Metropolis-Hastings sampling](https://github.com/AI4Finance-Foundation/RLSolver/blob/16bdc79fcefe1e044fe0b7ce7b837874f8035430/helloworld/maxcut/graph_max_cut_mcpg.py#L17) 2. 我想要知道这里被MCPG代码标记了 local search 的while loop里，为何有很多奇怪的常数？例如0.125 [while loop of local search](https://github.com/AI4Finance-Foundation/RLSolver/blob/16bdc79fcefe1e044fe0b7ce7b837874f8035430/helloworld/maxcut/graph_max_cut_mcpg.py#L65)

📝 Generate a fixed graph with a constant random seed

2

Generate a fixed undirected graph with a constant random seed 固定随机种子，生成固定的无向图 固定完随机种子之后，算法内部会生成固定的伪随机数， 随机生成无向图的函数 `generate_graph()`，会“消耗”这些伪随机数 下面的代码，可以在指定无向图的节点数量`num_nodes`，图的生成方式`g_type` 以及 图的序号`valid_i`，直接用代码生成固定的图。无论使用什么设备。 我建议使用方案1，避免储存很多表示无向图的txt文件（不用担心文件丢失）。只要指定以下三个信息，就能直接生成无向图，对于这样的 `graph_name = 'powerlaw_100_ID042'`，可以用函数直接生成唯一的无向图 ： - 无向图的节点数量`num_nodes=100` - 图的生成方式`g_type=‘powerlaw’` - 图的序号`valid_i=42` 备注：在生成300个节点的图，我们用的是方案3。在2023-08-17日之后才改成方案1 ---...