Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RuntimeError for action shape #91

Open
lz1159435992 opened this issue May 24, 2024 · 3 comments
Open

RuntimeError for action shape #91

lz1159435992 opened this issue May 24, 2024 · 3 comments

Comments

@lz1159435992
Copy link

I encountered issues with my previously working code after an update. I'm not sure what the source of the problem is, but there were no issues when using the version from a month ago. Can anyone tell me where I should focus my attention?

agent = PearlAgent(
                        policy_learner=SoftActorCritic(
                            state_dim=768,
                            action_space=action_space,
                            actor_hidden_dims=[768, 512, 128],
                            critic_hidden_dims=[768, 512, 128],
                            action_representation_module=action_representation_module,
                        ),
                        # history_summarization_module=StackingHistorySummarizationModule(
                        #     observation_dim=768,
                        #     action_dim=len(env.variables) + 1,
                        #     history_length=len(env.variables),
                        # ),
                        history_summarization_module=LSTMHistorySummarizationModule(
                            observation_dim=768,
                            action_dim=len(env.variables) + 1,
                            hidden_dim=768,
                            history_length=len(env.variables),  # 和完整结点数相同
                        ),
                        replay_buffer=FIFOOffPolicyReplayBuffer(10),
                        device_id=-1,
                    )
                    # 训练代理
                    info = online_learning(
                        agent=agent,
                        env=env,
                        number_of_episodes=number_of_episodes,
                        print_every_x_episodes=1,
                        record_period=record_period,
                        # learn_after_episode=True,
                    )
Traceback (most recent call last):
  File "/home/lz/PycharmProjects/Pearl/test_rl/test_group_gai_3_normal.py", line 181, in <module>
    test_group()
  File "/home/lz/PycharmProjects/Pearl/test_rl/test_group_gai_3_normal.py", line 150, in test_group
    info = online_learning(
  File "/home/lz/PycharmProjects/Pearl/test_rl/test_script/online_learning_break.py", line 115, in online_learning
    episode_info, episode_total_steps = run_episode(
  File "/home/lz/PycharmProjects/Pearl/test_rl/test_script/online_learning_break.py", line 285, in run_episode
    agent.observe(action_result)
  File "/home/lz/PycharmProjects/Pearl/pearl/pearl_agent.py", line 170, in observe
    new_subjective_state = self._update_subjective_state(action_result.observation)
  File "/home/lz/PycharmProjects/Pearl/pearl/pearl_agent.py", line 257, in _update_subjective_state
    return self.history_summarization_module.summarize_history(
  File "/home/lz/PycharmProjects/Pearl/pearl/history_summarization_modules/lstm_history_summarization_module.py", line 71, in summarize_history
    action = action.clone().detach().float().view((1, self.action_dim))
RuntimeError: shape '[1, 16]' is invalid for input of size 1
@rodrigodesalvobraz
Copy link
Contributor

I encountered issues with my previously working code after an update. I'm not sure what the source of the problem is, but there were no issues when using the version from a month ago. Can anyone tell me where I should focus my attention?

Can you send us a link to a repo we can clone and run your example? The snippet in your message does not define several of the variables, such as action_space, env and so on. I could try to figure those out from your previous messages, but that would be time-consuming and there would be the risk of me not getting exactly the same thing you're running.

@lz1159435992
Copy link
Author

https://github.com/lz1159435992/test_file.git
This is my example code repository. Running the file test_group_gai_3_normal.py under /test_rl will cause problems. The test files are placed in the /files directory, and the file path may be problematic, requiring specific modifications.
I apologize because my development is still in the early stages, and there is no documentation, and the code is quite messy. If you have any issues during the execution, please feel free to contact me at any time.

@rodrigodesalvobraz
Copy link
Contributor

rodrigodesalvobraz commented Jun 30, 2024

Hi. I finally got around to trying to reproduce this error, but then I hit line 10 in embedding_util.py which is
from pearl.SMTimer.KNN_Predictor import Predictor, and get
ModuleNotFoundError: No module named 'pearl.SMTimer'.
However, pearl does not have an SMTimer subpackage, so I am unable to proceed.

Ideally, if you could gradually reduce the code until you create a minimum error-reproducing example involving only Pearl code and showing the error occurring, that would likely be much easier for us and might also be clarifying for you as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants