pyluxcoretools netnode renderconfig errors

[nathanahtan]

I'm working on getting a render farm running.

After getting the pyluxcoretools to run, I'm finding an error with the netnode parsing config files from blender demo scenes:

std::bad_cast
Traceback (most recent call last):
  File "/home/nathan/lux/LuxCore/pyluxcoretools/renderfarm/renderfarmnode.py", line 168, in __HandleConnection
    config = pyluxcore.RenderConfig(renderConfigFile)
RuntimeError: std::bad_cast

I'm also getting a network error on the netconsole side:

Unable to receive a line
Traceback (most recent call last):
  File "/home/nathan/lux/LuxCore/./pyluxcoretools/renderfarm/renderfarmjobsingleimage.py", line 423, in NodeThread
    result = socketutils.RecvLine(nodeSocket)
  File "/home/nathan/lux/LuxCore/./pyluxcoretools/utils/socket.py", line 44, in RecvLine
    raise RuntimeError("Unable to receive a line")
RuntimeError: Unable to receive a line
Node thread done
Retrying node: 127.0.1.1:18018
Node thread started

Not sure if they are related. The file seems to transfer and rendering starts, but it doesn't get past the config error.

The scene renders in blender.

I've tried different scenes and changed render settings, but all result in the config error, so I don't think it is scene dependent.

Though I'd ask for advice on the best way to find out what the issue is.

[nathanahtan]

Update I can render the same .bcf file with the console, so it must be in the network layer.

[Dade916]

What LuxCore version are you using ?

[nathanahtan]

Linux, V2.5, I guess? Are you looking for a more specific version number? Is there a way to get it? I downloaded from https://github.com/LuxCoreRender/LuxCore/releases/download/luxcorerender_v2.5/luxcorerender-v2.5-linux64-sdk.tar.bz2 the other day.

[nathanahtan]

Perhaps I have something else wrong because I've hit some other issues:

Trying to access the network manager from blender(2.92.0), it says I need to install PySide, but I think whatever interpreter (I think the system python3) it is trying to install into is too new and PySide won't install. Maybe this needs to use PySide2 or PySide6 now?

In reference to my above comment 'after getting pyluxcoretools to run':

I found launching the menu, netnodeui or netconsoleui from luxcoretools.zip crashed. However, the non-ui tools seemed to work. After some investigation, it seemed to be an import issue in __main__.py. I had to move all the pycoretools imports into the arg parsing if/elif statement:

if generalArgs.commandToExecute == "help":
	generalParser.print_help()
elif generalArgs.commandToExecute == "console":
	import pyluxcoretools.pyluxcoreconsole.cmd as consoleCmd
	consoleCmd.main(cmdArgv)
elif generalArgs.commandToExecute == "merge":
	import pyluxcoretools.pyluxcoremerge.cmd as mergeCmd
	mergeCmd.main(cmdArgv)
elif generalArgs.commandToExecute == "netconsole":
	import pyluxcoretools.pyluxcorenetconsole.cmd as netConsoleCmd
	netConsoleCmd.main(cmdArgv)
elif generalArgs.commandToExecute == "netconsoleui":
	import pyluxcoretools.pyluxcorenetconsole.ui as netConsoleUI
	netConsoleUI.main(cmdArgv)
elif generalArgs.commandToExecute == "netnode":
	import pyluxcoretools.pyluxcorenetnode.cmd as netNodeCmd
	netNodeCmd.main(cmdArgv)
elif generalArgs.commandToExecute == "netnodeui":
	import pyluxcoretools.pyluxcorenetnode.ui as netNodeUI
	netNodeUI.main(cmdArgv)
elif generalArgs.commandToExecute == "menu":
	import pyluxcoretools.pyluxcoremenu.cmd as menuCmd
	menuCmd.main(cmdArgv)
else:
	raise TypeError("Unknown command to execute: " + generalArgs.commandToExecute)

It seems like if any of the .ui modules get imported, while any of the other modules are already imported, a crash occurs.

[Dade916]

Can you try the under development v2.6 (https://forums.luxcorerender.org/viewtopic.php?f=9&t=736) ?

P.S. pyluxcoretools require PySide2. It may work with PySide1 but, as you have seen, it pretty much not supported by systems.

[nathanahtan]

UI's started. Tried to load same binary scene. Same recvline error on netconsole:

[MainThread][2021-11-29 15:16:36,799] LuxCore 2.6alpha0
[NetBeaconReceiverThread][2021-11-29 15:16:36,800] NetBeaconReceiver thread started.
[NetBeaconReceiverThread][2021-11-29 15:16:43,122] Discovered new node: 127.0.1.1:18018
[MainThread][2021-11-29 15:17:05,176] Creating single image render farm job: /home/nathan/lux/LuxCore/scenes/LuxBall/export/luxball_LuxCore/00042.bcf
[MainThread][2021-11-29 15:17:05,176] New render farm job: /home/nathan/lux/LuxCore/scenes/LuxBall/export/luxball_LuxCore/00042.bcf
[MainThread][2021-11-29 15:17:05,195] Job file md5: 15b299498d9c7bd4daa667b078483595
[MainThread][2021-11-29 15:17:05,195] Previous scene MD5: 15b299498d9c7bd4daa667b078483595
[MainThread][2021-11-29 15:17:05,195] Current scene MD5:  15b299498d9c7bd4daa667b078483595
[MainThread][2021-11-29 15:17:05,195] /home/nathan/lux/LuxCore/scenes/LuxBall/export/luxball_LuxCore/00042-netrendering exists and is valid. Continue the rendering
[MainThread][2021-11-29 15:17:05,195] Merging all previous films
[MainThread][2021-11-29 15:17:05,195] -------------------------------------------------------
[MainThread][2021-11-29 15:17:05,195] Job started: /home/nathan/lux/LuxCore/scenes/LuxBall/export/luxball_LuxCore/00042.bcf
[MainThread][2021-11-29 15:17:05,195] -------------------------------------------------------
[RenderFarmNodeThread-127.0.1.1:18018][2021-11-29 15:17:05,196] Node thread started
[FilmMergeThread][2021-11-29 15:17:05,196] Film merge thread started
[RenderFarmNodeThread-127.0.1.1:18018][2021-11-29 15:17:05,197] Remote node has the same pyluxcore verison
[RenderFarmNodeThread-127.0.1.1:18018][2021-11-29 15:17:05,197] Sending file: /home/nathan/lux/LuxCore/scenes/LuxBall/export/luxball_LuxCore/00042.bcf
[RenderFarmNodeThread-127.0.1.1:18018][2021-11-29 15:17:05,205] Transfered 5.09 Mbytes in 00:00:00 (922.93 Mbytes/sec)
[RenderFarmNodeThread-127.0.1.1:18018][2021-11-29 15:17:05,205] Sending seed: 18
[RenderFarmNodeThread-127.0.1.1:18018][2021-11-29 15:17:05,205] Waiting for node rendering start
[RenderFarmNodeThread-127.0.1.1:18018][2021-11-29 15:17:05,364] Unable to receive a line
Traceback (most recent call last):
  File "pyluxcoretools.zip/pyluxcoretools/renderfarm/renderfarmjobsingleimage.py", line 423, in NodeThread
    result = socketutils.RecvLine(nodeSocket)
  File "pyluxcoretools.zip/pyluxcoretools/utils/socket.py", line 44, in RecvLine
    raise RuntimeError("Unable to receive a line")
RuntimeError: Unable to receive a line
[RenderFarmNodeThread-127.0.1.1:18018][2021-11-29 15:17:05,365] Node thread done

But now the netnode crashes once starting:

[MainThread][2021-11-29 15:16:18,016] LuxCore 2.6alpha0
[NetBeaconSenderThread][2021-11-29 15:16:43,121] NetBeaconSender thread started.
[Thread-1][2021-11-29 15:16:43,122] Waiting for a new connection
[Thread-1][2021-11-29 15:17:05,196] Received connection from: ('127.0.0.1', 39048)
[Thread-1][2021-11-29 15:17:05,196] Remote pyluxcore version: 2.6alpha0
[Thread-1][2021-11-29 15:17:05,197] Local pyluxcore version: 2.6alpha0
[Thread-1][2021-11-29 15:17:05,197] Receiving RenderConfig serialized file: renderfarmnode-8ad3c658-166f-45ce-b93d-3e6a43db730a.bcf
[Thread-1][2021-11-29 15:17:05,197] Receiving file: renderfarmnode-8ad3c658-166f-45ce-b93d-3e6a43db730a.bcf
[Thread-1][2021-11-29 15:17:05,205] Transfered 5.09 Mbytes in 00:00:00 (666.44 Mbytes/sec)
[Thread-1][2021-11-29 15:17:05,214] Receiving RenderConfig serialized MD5: 15b299498d9c7bd4daa667b078483595
[Thread-1][2021-11-29 15:17:05,215] Received seed: 18
[Thread-1][2021-11-29 15:17:05,215] Reading RenderConfig serialized file: renderfarmnode-8ad3c658-166f-45ce-b93d-3e6a43db730a.bcf
[Thread-1][2021-11-29 15:17:05,215] [SDL][] Define ImageMap: NamedObject
Segmentation fault (core dumped)

But also so does luxcoreui on the same scene file (exported from blender)

I tried luxcoreui on the scene included with the latest release and it worked. Exported as binary so I can load it into the netconsole ui as a job. But, I see now that the 'binary' is a zip file (not a .bcf). Tried extracting the zip and renaming result to .bcf. netconsole sent it but now the netnode complains about gzip compression:

Reading RenderConfig serialized file: renderfarmnode-1c6643bb-01ae-47be-b849-304502243849.bcf
gzip error: iostream error
Traceback (most recent call last):
  File "pyluxcoretools.zip/pyluxcoretools/renderfarm/renderfarmnode.py", line 168, in __HandleConnection
    config = pyluxcore.RenderConfig(renderConfigFile)
RuntimeError: gzip error: iostream error
Connection done: ('127.0.0.1', 39066)

Does netconsoleui need to be updated to load zip files?

[Dade916]

Have you re-created the .bcf file with v2.6 ? Binary files are not portable across different versions.

[nathanahtan]

OK. Not good news I'm afraid. Steps taken:

1. Download Blender 2.93.6.
2. Download 'Latest' luxrender and blender add on.
3. Try to launch luxcore network render manager from blender...same PySide issue.
4. Load a demo scene, check it is rendering...export as binary file.
5. Load binary scene with luxcoreui -> Renders.
6. Launch netconsoleui from luxcoretools.zip -> Fixed, worked (I see someone else discovered the import issue already)
7. Launch netnodeui from luxcoretools.zip -> Fixed.
8. Start netnode....connects, seems OK.
9. Load binary scene file....file transfer ok.....render starts.....then:

[SDL][] Texture definition: 140679337346568Color
std::bad_cast
Traceback (most recent call last):
  File "pyluxcoretools.zip/pyluxcoretools/renderfarm/renderfarmnode.py", line 168, in __HandleConnection
    config = pyluxcore.RenderConfig(renderConfigFile)
RuntimeError: std::bad_cast
Connection done: ('127.0.0.1', 39420)

On the netnode.

So this is basically back to the original issue with the stable version.

Any ideas?

[Dade916]

Do you have some kind of no-US/English locale (language setting) ?

[nathanahtan]

Not that I'm aware of. I'm in the US. Installed Ubuntu with US/English and that's what it says in my language settings.

[Dade916]

Does it happen with all scenes or only with a specific one ? Even with "Cornell" scene ?

[nathanahtan]

All scenes so far. Yes, Even the Cornell one.