-
Notifications
You must be signed in to change notification settings - Fork 35
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Inquiry About "4d_humans" Folder in NeuMan Dataset #6
Comments
Have you made any progress? I have the same question here. @TCN1214 |
Anyone who is interested in motion capture can Joint wechat group through me. my wechat id is Denny0805789 |
8.9 Edit 7.9 Edit Using neuman's preprocessing process, you should get files other than smpl_optimised_aligned_scale.npz (sensepose,depthmaps,images,keypoints,monodepth,segments,smpl_pred ,sparse, etc.). In the preprocessing flow, change the function for solve_transformation in export_alignments.py to:
Also remember to save the results of scale Human pose prediction using PHALP as mentioned by 4D-humans. Take the last line of alignments.npy as transl
then u will get smpl_optimised_aligned_scale.npz |
@ZCWzy were you able to resolve this issue and train humans on your own data? |
@yashgarg98 no, There is still a problem with the global_orient parameter. |
Hi, @ZCWzy, you mentioned that the body_pose in smpl_optimized_aligned_scale.npz comes from HMR_results (4D-humans), right? But when I print them out, they appear in different formats. For example, for per frame, The body_pose in 4D humans: The body_pose in smpl_optimized_aligned_scale.npz : So, do you use any methods to convert them? |
@TCN1214 The body_pose in 4D-humans is a rotation matrix representation and the body_pose in smpl_optimized_aligned_scale.npz is an axis angle representation (Axis with angle magnitude). |
@ZCWzy, got it, thanks! And may I know if you also get the global_orient and betas from 4D Human? I noticed that in smpl_optimized_aligned_scale.npz , the betas are the same for all frames. Do you have any idea why? |
4d-Humans tracking is demonstrated with the help of PHALP. The bbox, global_orient, body_pose and betas can be obtained directly using PHALP. But there's a problem with global_orient in this case, it doesn't work directly. beta is used to describe the shape of the human body, and the authors may have assumed that the shape of the subject's body would not change |
The betas produced from 4D human data are different for each frame. May I know how I can get the same betas for all frames? |
I simply averaged |
Alright, I got it. Thanks for your answer! |
Hi, @ZCWzy, you mentioned that scale and translation come from alignments.npy right? But when I compare it, I noticed that the translation values in smpl_optimized_aligned_scale.npz are different from alignment.npy, And also, the scale I obtained from alignment.npy is also different from the smpl_optimized_aligned_scale.npz, The scale I obtained from alignment.npy (parkinglot from Neuman dataset) So could you tell me more details about how you get the scales and translation? Do you just run alignment.npy or do you add some modifications? |
@TCN1214 scale has a more accurate way of fetching(from neuman/export_alignment)Numerical errors are quite normal. Every time you run export_alignment.py the scale is different |
Alright, thank you for your help! |
@ZCWzy May I know how you separate them? |
I'm not sure because I don't multi view geometry. You should refer to neuman/export_alignment.py, as well as the relevant papers to be sure |
Alright, I'll look into them. Thank you for your answer! |
@ZCWzy may I know what is your structure of the open3d_registration function, and what is it used for? |
open3d_registration is a method of the open source library open3d for the purpose of point cloud alignment. Here I want to move the original verts (which is a point cloud of 6890 points) to verts_world by scaling and translating it, so that the two point clouds are only rotated. In this way the global_orient is corrected to the correct position. But there seems to be a problem with this line of thinking, |
@ZCWzy Sorry, I'm not sure yet.... Now I'm trying your method to generate the global_orient. May I know where I can refer to the code of open3d_registration function? |
@TCN1214 use this:
then
That seems about right. |
@ZCWzy You mean you have an error, right? Can I have a look at your error? |
@ZCWzy I have encountered this error before. The reason I didn't satisfy the condition assert x.max() <= 1 + EPS and x.min() >= -EPS, f "x must be in [0, 1], got {x.min()} and {x.max()}" was because I used the wrong scale and translation. When I used the translation and scale values from alignments.npy, the problem was solved. However, I don't think the reason you're getting this error is the same as mine, because you used the translation and scale values from alignments.npy, right? |
I remove this assert then train. It looks like there's no problem. |
@ZCWzy When I run the command !python track.py video.source=(your images path) in the Colab provided by 4D Humans, I get this error: [12/03 09:12:56] INFO No OpenGL_accelerate module loaded: No module When I downloaded OpenGL_accelerate, I get this error below: [07/09 10:29:53] INFO OpenGL_accelerate module loaded This is why? |
put basicModel_neutral_lbs_10_207_0_v1.0.0.pkl in /4D-humans/data and /4D-humans |
@ZCWzy Yeah, I put the model already, but It still has this error (INFO No OpenGL_accelerate module loaded: No module), so I think the problem is not the SMPL model. Are you able to run the colab provided by 4D humans? |
https://colab.research.google.com/drive/11XZBoaMfa28y874r4B7VSVokNFlXiqqz?usp=sharing |
Hey, @ZCWzy after the alignment, were you able to fix the global coordinate issue? Also are you able to generate the human gaussian splat using HUGS? |
Hi @ZCWzy, I am currently encountering some difficulties while preprocessing my dataset, specifically with the 4dhuman folder. I noticed that you successfully processed your data. Could you please share your preprocessing code? |
Yeah, but my rebuild didn't work out so well. |
这位北大的✌好,请参照我之前的回复产生自定义数据。bbox,bodypose,betas都很容易利用4d-humans生成。global_orient,scale和transl需要轻微魔改一下neuman的预处理代码。这个数据预处理和转换不难写的 |
Can you share some of the images that you reconstructed using your data to check the quality of the images? |
My workstation was sent in for repair and I can't offer it at the moment. I have a feeling that the face alignment accuracy of the model isn't quite accurate enough, or that the quality of my images is too poor |
@ZCWzy 你好兄弟,可以给个联系方式吗 想问一下如何预处理自定义数据,并跑通它 |
看我新仓库() |
你好,请问你解决了吗,希望得到你的帮助,谢谢 |
你好,请问那个4d-huamns文件夹你完成了吗,希望得到你的帮助 |
你可以参考 @ZCWzy 个人主页里的仓库来完成。不过我个人感觉ml-hugs的代码对于自己的数据集不太友好,所以我后来没有使用这篇工作。祝好! |
ZCWzy我已经试过了,可能训练的效果不是太好,所以想问你一下,抱歉打扰了 |
你好,我想在问你个问题,你试过多人放在一个场景中吗? |
Hi, thanks for releasing the code. When I downloaded your NeuMan dataset, I noticed an additional folder called "4d_humans," which isn't present in the original NeuMan dataset. Could you let me know where you obtained the files in the "4d_humans" folder, such as smpl_optimized_aligned_scale.npz, smpl_aligned_scale.npz, poses_optimized.npz, and poses.npz?
The text was updated successfully, but these errors were encountered: