Using the results as input of Gaussian Splatting trainning data

[DabblerGISer]

I'm currently working on combine the mast3r sfm pipeline based on what's given in kapture_mast3r_mapping.py(see branch mast3r_sfm) with gaussian splatting training.
I found that although the result of the sfm resconstruction is pretty good but the result of gaussian splatting shows the psnr is not satisfying. Anyone have some ideas?
I guess it might because of the initial distribution of the point cloud. I tried to densify the point cloud but the psnr is not improved. Could it be influenced by the initial distribution of the point cloud?

[DabblerGISer]

I tried to increase the pairs the output seems improved.

[denghilbert]

Hi @DabblerGISer,

Can you share the command and arguments that you use for preparing data for GS? I am also trying to use this to obtain an init for GS. It will be really helpful if you can share. Thanks!

[DuVogel87]

@DabblerGISer
I would also be interested to know how you prepare the data for splat training.
I am used to use Colmap and then train from that.
Could you also share what code you use for training?
Cheers!

[Mlynnray]

hello, @DabblerGISer
I'm also prepare to use it for training splat, but I encounter some questions,if you can share your experence it would help me a lot.Thank you!

[DabblerGISer]

hello, @DabblerGISer I'm also prepare to use it for training splat, but I encounter some questions,if you can share your experence it would help me a lot.Thank you!

@DabblerGISer I would also be interested to know how you prepare the data for splat training. I am used to use Colmap and then train from that. Could you also share what code you use for training? Cheers!

Hi @DabblerGISer,

Can you share the command and arguments that you use for preparing data for GS? I am also trying to use this to obtain an init for GS. It will be really helpful if you can share. Thanks!

Good afternoon guys. You could check the branch masr3r_sfm which contains the code of sfm following colmap format, meanwhile also the checkpoints needed to generate. Firstly generate pairs.txt file using "make_pairs.py" or you can generate pairs file by your own algorithm corresponding to this structure: 'nameA.jpg nameB.jpg 1'. And then run the kapture_mast3r_mapping.py to get the sfm result. There are some bugs needed to be amended, some are mentioned in the issues check this: #99 (comment)
And you also might need to undistort the images afterwards.
If you have any questions please let me know

[Mlynnray]

@DabblerGISer Since I am not a computer science major, I am sorry to bother you again. When I run make_pairs.py, I set the path of retrieval_model to the path of weights, but it obviously reports an error, so can you tell me how to set this?

[DabblerGISer]

@DabblerGISer Since I am not a computer science major, I am sorry to bother you again. When I run make_pairs.py, I set the path of retrieval_model to the path of weights, but it obviously reports an error, so can you tell me how to set this?

python make_pairs.py --dir /the/dir/containing/your/photos --output /the/path/to/your/pairs.txt --retrieval_model MASt3R_ViTLarge_BaseDecoder_512_catmlpdpt_metric_retrieval_trainingfree.pth --weights MASt3R_ViTLarge_BaseDecoder_512_catmlpdpt_metric.pth I think this should be correct.

[oUp2Uo]

Hi @DabblerGISer ,

Do you ever meet error like this:

Traceback (most recent call last):
  File "D:\workspace\mast3r\make_pairs.py", line 100, in <module>
    main(args.dir, args.scene_graph, args.output, backbone, retrieval_model)
  File "D:\workspace\mast3r\make_pairs.py", line 73, in main
    pairs = make_pairs(imgs, scene_graph, prefilter=None, symmetrize=True, sim_mat=sim_matrix)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\workspace\mast3r\mast3r\image_pairs.py", line 59, in make_pairs
    mode, Na, k = scene_graph.split('-')
    ^^^^^^^^^^^
ValueError: too many values to unpack (expected 3)

I tried to add --scene_graph complete, add then make_pairs.py could run.
And kapture_mast3r_mapping.py could get the result.

Btw, how do you succeed to 'increase the pairs the output'?

[TamandTat]

Hi @DabblerGISer , 嗨@DabblerGISer，

Do you ever meet error like this:您是否遇到这样的错误：

Traceback (most recent call last):
  File "D:\workspace\mast3r\make_pairs.py", line 100, in <module>
    main(args.dir, args.scene_graph, args.output, backbone, retrieval_model)
  File "D:\workspace\mast3r\make_pairs.py", line 73, in main
    pairs = make_pairs(imgs, scene_graph, prefilter=None, symmetrize=True, sim_mat=sim_matrix)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\workspace\mast3r\mast3r\image_pairs.py", line 59, in make_pairs
    mode, Na, k = scene_graph.split('-')
    ^^^^^^^^^^^
ValueError: too many values to unpack (expected 3)

I tried to add --scene_graph complete, add then make_pairs.py could run. And kapture_mast3r_mapping.py could get the result.我尝试添加--scene_graph complete，然后添加make_pairs.py可以运行。 kapture_mast3r_mapping.py可以得到结果。

Btw, how do you succeed to 'increase the pairs the output'?顺便说一句，您如何成功地“增加输出对”？

I've also encountered this problem. Have you managed to fix it? Could you share how you did it?