Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Libraries claiming to be extracted but not working #37

Closed
siddhartha-gadgil opened this issue Nov 3, 2017 · 11 comments
Closed

Libraries claiming to be extracted but not working #37

siddhartha-gadgil opened this issue Nov 3, 2017 · 11 comments

Comments

@siddhartha-gadgil
Copy link

I was trying to experiment with tensorflow_scala, but I get an error when running a tiny test program (in sbt) after a separate tensorflow installation

2017-11-03 10:35:30.468 [run-main-0] INFO  TensorFlow Native - Extracting the 'tensorflow_jni' native library to /tmp/tensorflow_scala_native_libraries1715329719143383324/libtensorflow_jni.so.
2017-11-03 10:35:30.475 [run-main-0] INFO  TensorFlow Native - Copied 628432 bytes to /tmp/tensorflow_scala_native_libraries1715329719143383324/libtensorflow_jni.so.
2017-11-03 10:35:30.476 [run-main-0] INFO  TensorFlow Native - Extracting the 'tensorflow_ops' native library to /tmp/tensorflow_scala_native_libraries1715329719143383324/libtensorflow_ops.so.
2017-11-03 10:35:30.477 [run-main-0] INFO  TensorFlow Native - Copied 48536 bytes to /tmp/tensorflow_scala_native_libraries1715329719143383324/libtensorflow_ops.so.
[error] (run-main-0) java.lang.ExceptionInInitializerError
java.lang.ExceptionInInitializerError
	at org.platanios.tensorflow.jni.Tensor$.<init>(Tensor.scala:24)
	at org.platanios.tensorflow.jni.Tensor$.<clinit>(Tensor.scala)
	at org.platanios.tensorflow.api.tensors.Context$.apply(Context.scala:50)
	at org.platanios.tensorflow.api.package$.<init>(package.scala:47)
	at org.platanios.tensorflow.api.package$.<clinit>(package.scala)
	at example.Hello$.delayedEndpoint$example$Hello$1(Hello.scala:9)
	at example.Hello$delayedInit$body.apply(Hello.scala:6)
	at scala.Function0.apply$mcV$sp(Function0.scala:34)
	at scala.Function0.apply$mcV$sp$(Function0.scala:34)
	at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
	at scala.App.$anonfun$main$1$adapted(App.scala:76)
	at scala.collection.immutable.List.foreach(List.scala:389)
	at scala.App.main(App.scala:76)
	at scala.App.main$(App.scala:74)
	at example.Hello$.main(Hello.scala:6)
	at example.Hello.main(Hello.scala)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
Caused by: org.platanios.tensorflow.jni.NotFoundException: /tmp/tensorflow_scala_native_libraries1715329719143383324/libtensorflow_ops.so: undefined symbol: _ZN10tensorflow7strings8internal9CatPiecesESt16initializer_listINS_11StringPieceEE
	at org.platanios.tensorflow.jni.TensorFlow$.loadOpLibrary(Native Method)
	at org.platanios.tensorflow.jni.TensorFlow$.$anonfun$load$8(TensorFlow.scala:138)
	at scala.Option.foreach(Option.scala:257)
	at org.platanios.tensorflow.jni.TensorFlow$.load(TensorFlow.scala:138)
	at org.platanios.tensorflow.jni.TensorFlow$.<init>(TensorFlow.scala:186)
	at org.platanios.tensorflow.jni.TensorFlow$.<clinit>(TensorFlow.scala)
	at org.platanios.tensorflow.jni.Tensor$.<init>(Tensor.scala:24)
	at org.platanios.tensorflow.jni.Tensor$.<clinit>(Tensor.scala)
	at org.platanios.tensorflow.api.tensors.Context$.apply(Context.scala:50)
	at org.platanios.tensorflow.api.package$.<init>(package.scala:47)
	at org.platanios.tensorflow.api.package$.<clinit>(package.scala)
	at example.Hello$.delayedEndpoint$example$Hello$1(Hello.scala:9)
	at example.Hello$delayedInit$body.apply(Hello.scala:6)
	at scala.Function0.apply$mcV$sp(Function0.scala:34)
	at scala.Function0.apply$mcV$sp$(Function0.scala:34)
	at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
	at scala.App.$anonfun$main$1$adapted(App.scala:76)
	at scala.collection.immutable.List.foreach(List.scala:389)
	at scala.App.main(App.scala:76)
	at scala.App.main$(App.scala:74)
	at example.Hello$.main(Hello.scala:6)
	at example.Hello.main(Hello.scala)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
[trace] Stack trace suppressed: run last compile:run for the full output.
java.lang.RuntimeException: Nonzero exit code: 1
	at scala.sys.package$.error(package.scala:27)
[trace] Stack trace suppressed: run last compile:run for the full output.
[error] (compile:run) Nonzero exit code: 1
[error] Total time: 0 s, completed 3 Nov, 2017 10:35:30 AM

I checked to see that Extracting the 'tensorflow_jni' native library to /tmp/tensorflow_scala_native_libraries1715329719143383324/libtensorflow_jni.so did not actually happen, i.e. there is no sub-directory in /tmp with the above (or any similar name).

I am using Ubuntu 16.04. I have checked that Tensorflow works with Python.

Any help is appreciated
@eaplatanios
Copy link
Owner

@siddhartha-gadgil The libraries are extracted fine but are deleted once the program terminates. That's why you don't see them in the tmp directory. The problem in your case has to do with which version of the compiled TensorFlow binaries you're working with. Could you try setting the following dependency in your build.sbt file?

libraryDependencies += "org.platanios" %% "tensorflow" % "0.1.0-SNAPSHOT" classifier "linux-cpu-x86_64"

This will pull the pre-compiled binaries I distribute with my releases. Otherwise, not that for now, if you want to provide your own binaries, you should compile TensorFlow yourself using master branch from the official TensorFlow repository, or you should download the nightly versions of their pre-compiled binaries.

Please let me know if this helps. :)

@siddhartha-gadgil
Copy link
Author

siddhartha-gadgil commented Nov 3, 2017 via email

@eaplatanios
Copy link
Owner

@siddhartha-gadgil It's not an issue with your configuration. I was able to re-produce the problem and I'm currently looking into how to fix it. I'll update you soon. Thanks for finding that! :)

@eaplatanios
Copy link
Owner

@siddhartha-gadgil I think the issue has been fixed with commit #fc0428d, but please let me know if you still get the same error.

@siddhartha-gadgil
Copy link
Author

siddhartha-gadgil commented Nov 6, 2017 via email

@eaplatanios
Copy link
Owner

@siddhartha-gadgil As a first step I would suggest you remove your TensorFlow installation (the so files) and use this dependency on a test project:

libraryDependencies += "org.platanios" %% "tensorflow" % "0.1.0-SNAPSHOT" classifier "linux-cpu-x86_64"

You should also clear your Ivy cache before so you get the latest version of my API. You can do this (usually) by deleting the ~/.ivy2/cache/org.platanios directory.

Could you please tell me if this works? And then we can see what version of the TensorFlow source you should use.

@siddhartha-gadgil
Copy link
Author

Works :) Thanks a lot.
I am happy using the classifier since for now I am just experimenting.

@eaplatanios
Copy link
Owner

@siddhartha-gadgil That's great! :) So, whenever you get the chance, you could try compiling with the master branch of TensorFlow repository. That should normally work. If it doesn't, I'd more than happy to help figure out what's wrong. :)

@siddhartha-gadgil
Copy link
Author

siddhartha-gadgil commented Nov 6, 2017 •
edited
Loading

I had compiled with the master branch(CPU only) and got the issue reported above. It seems the same as tensorflow/fold#57

@eaplatanios
Copy link
Owner

Given that I recently made changes to how the libraries are loaded. You could try again now that you cleared your Ivy cache and it might work. I'm actually curious.

@siddhartha-gadgil
Copy link
Author

The error is back. In case this is relevant, while compliling in sbt (after clean) I get the warning

[info] Resolving org.platanios#tensorflow_2.12;0.1.0-SNAPSHOT ...
[warn] Unable to reparse org.platanios#tensorflow_2.12;0.1.0-SNAPSHOT from sonatype-snapshots, using Mon Nov 06 05:13:23 IST 2017
[info] Resolving org.platanios#tensorflow-jni_2.12;0.1.0-SNAPSHOT ...
[warn] Unable to reparse org.platanios#tensorflow-jni_2.12;0.1.0-SNAPSHOT from sonatype-snapshots, using Mon Nov 06 05:11:30 IST 2017
[info] Resolving org.platanios#tensorflow-api_2.12;0.1.0-SNAPSHOT ...
[warn] Unable to reparse org.platanios#tensorflow-api_2.12;0.1.0-SNAPSHOT from sonatype-snapshots, using Mon Nov 06 05:13:55 IST 2017

@eaplatanios eaplatanios mentioned this issue Nov 6, 2017
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@eaplatanios @siddhartha-gadgil