Difficult with dependecies using Python in my RCP app

Tell us how you're using Python Support in NetBeans IDE to build...

Difficult with dependecies using Python in my RCP app

Postby bbergeson » Tue Feb 09, 2016 7:59 pm

I already sent my question using the email address but I don't see it on this forum so I am posting it again using this method. I am running NetBeans 8.1 and jdk 1.8 update 72 64bit. Here are the warnings I am getting:

Warning - could not install some modules:
Python Core - The module named org.netbeans.modules.project.ant/1 was needed and not found
Python Editor - The module named org.netbeans.modules.gsf.codecoverage was needed and not found
Python Project - The module named org.openidex.util/3 was needed and not found
Python Project - The module named org.netbeans.modules.gsf.codecoverage was needed and not found
Python Project - The module named org.netbeans.modules.gototest/1 was needed and not found
Python Project - The module named org.netbeans.modules.project.ant/1 was needed and not found
Python Testrunner - The module named org.netbeans.modules.gsf.testrunner/2 was needed and not found
Python Testrunner - The module named org.netbeans.modules.project.ant/1 was needed and not found
Python Testrunner - The module named org.netbeans.modules.gsf/testrunner.ui was needed and not found
11 further modules couldnot be installed due to the above problems.

I am considering contributing to this project since Python support in NetBeans is very important to all of my customers that use my RCP apps created with NetBeans. If I can get this working and it works well then I will more than likely be willing to contribute.

Thx Blaine
bbergeson
 
Posts: 10
Joined: Thu Feb 04, 2016 5:01 pm

Re: Difficult with dependecies using Python in my RCP app

Postby mr_lou_d » Sat Feb 20, 2016 8:05 pm

Some thoughts,

We currently build the Python plugin with JDK 7u79, and may migrate to JDK 9 (?) by this Summer or thereabouts. It depends on what NetBeans.org decides to do for IDE 8.2

If you can submit (or push to a public repo) a short sample RCP app with Python modules as you intend to build it, Julien and I can review it.

Regards, Lou
User avatar
mr_lou_d
Site Admin
 
Posts: 237
Joined: Thu May 22, 2014 11:36 am
Location: Chicago, IL, USA

Re: Difficult with dependecies using Python in my RCP app

Postby bbergeson » Thu Jul 21, 2016 8:21 pm

Okay, I finally have time to get back to this issue which is still unresolved so let me recap. I am running on Windows 7 64bit. I am not using java 1.8 update 92 64 bit. I am using NetBeans 8.1. I installed the NBPython plugin by first going to Settings in the Plugins dialog and changing the Plugin Install Location to "Force install into shared directories" and then I install the NBPython plugin. Next I create a new "NetBeans Platform Application." I then right click on the new project and select properties and then go to the libraries tab. I expand the "extra" group and start checking specific Python related modules. I can check Python Console, Python Core, and Python Debugger without any problems. As soon as I select Python Editor I get the following:

Module Python Editor in extra depends on module org.eclipse.mylyn.wikitext.confluence.core in ide but this is excluded.

If I enable org.eclipse.mylyn.wikitext.confluence.core in ide I get the following:

Module org.eclipse.mylyn.wikitext.confluence.core in ide requests the token org.eclipse.mylyn.wikitext.core but all providers (e.g. module org.eclipse.mylyn.wikitext.core in ide) are excluded.

So I enable that and I get the following:

Module org.eclipse.mylyn.wikitext.confluence.core in ide requests the token org.eclipse.team.core but there are no known providers.

I am now stuck. I have provided enough details that anyone should be able to reproduce this.
bbergeson
 
Posts: 10
Joined: Thu Feb 04, 2016 5:01 pm

Re: Difficult with dependecies using Python in my RCP app

Postby bbergeson » Wed Aug 24, 2016 8:22 pm

Okay, I have it working. I ended up having to enable most of the modules in the ide platform. I really only needed the editor support because I am using a script engine to run python scripts. Unfortunately because of all the dependencies this pluggin has, I had to enable many more libraries than I actually use. I would suggest organizing the code so that a user could just enable the editing features without requiring all the project, run, debug, etc. features.
bbergeson
 
Posts: 10
Joined: Thu Feb 04, 2016 5:01 pm

Re: Difficult with dependecies using Python in my RCP app

Postby bbergeson » Fri Aug 26, 2016 7:37 pm

It looks like I spoke too soon. It works when I run my app within NetBeans but when I run it outside of NetBeans I get a pop up that indicates that I am still missing some dependent libraries such as:

org.apache.ant - I found some libraries on the Apache ANT website but the package path for all of them are org.apache.tools.ant which is not the same.
org.apache.xerces - I thought I found one for this one but when I placed it in the same directory as the next once I still didn't see it in the list of modules under extra.
org.eclipse.team.org

I was able to locate a jar file for the last one on the internet. I downloaded it and placed it in the extra/modules directory and then it showed up in the list of modules. I have not been able to find jar files for the first two shown above.

This doesn't make sense to me. Why can I run NetBeans with this pluggin without any problems yet when I build my own app and include this pluggin I am missing some dependencies. The dependent files must be part of NetBeans otherwise it shouldn't work either. I am having to spend too much time on this but I have to get it working.

Arrrggg
bbergeson
 
Posts: 10
Joined: Thu Feb 04, 2016 5:01 pm

Re: Difficult with dependecies using Python in my RCP app

Postby Jenselme » Thu Sep 01, 2016 7:54 am

bbergeson wrote:It looks like I spoke too soon. It works when I run my app within NetBeans but when I run it outside of NetBeans I get a pop up that indicates that I am still missing some dependent libraries such as:

org.apache.ant - I found some libraries on the Apache ANT website but the package path for all of them are org.apache.tools.ant which is not the same.
org.apache.xerces - I thought I found one for this one but when I placed it in the same directory as the next once I still didn't see it in the list of modules under extra.
org.eclipse.team.org


Strange, I cannot think of a reason why it would work when run with NetBeans but not when run outside.

For the missing deps, I think that for ant you can use for ant http://hg.netbeans.org/binaries/077C746ECAB048B9839C7A8E39E55FE8636C5B11-ant.jar and for exerces http://hg.netbeans.org/binaries/9BB329DB1CFC4E22462C9D6B43A8432F5850E92C-xerces-2.11.0.jar. You after that you are still missing libraries, you should look at http://hg.netbeans.org/binaries/ where you will find the list of all binaries used by netbeans (the page may take a while to load, there's lots of binaries).

bbergeson wrote:Unfortunately because of all the dependencies this pluggin has, I had to enable many more libraries than I actually use. I would suggest organizing the code so that a user could just enable the editing features without requiring all the project, run, debug, etc. features.


This would be great indeed. I think this would require a big refactoring and huge changes in how the Python plugins are created and built. Since the team is small and from my very small experience with dealing with that, it would require a significant amount of time. So I don't think it is likely to happen soon.
Jenselme
 
Posts: 116
Joined: Thu Apr 16, 2015 9:05 pm
Location: Paris, France

Re: Difficult with dependecies using Python in my RCP app

Postby bbergeson » Thu Sep 01, 2016 3:07 pm

I don't think I was clear in my last statement. When I said that it works in NetBeans it actually still gives me warnings about the missing dependencies but I am able to run in Debug or Run mode without any problems. When I run outside of NetBeans the same warnings cause a pop up and requires me to disable the affected modules or exit. So running outside of NetBeans is doesn't let me ignore the warnings and if I choose to disable the modules and continue then the python plugin doesn't fully work.

I tried downloading the two libraries you provided but NetBeans doesn't seem to recognize them, meaning that when I put them in the NetBeans 8.1/extra/modules directory I don't see them show up under the extra node in the suite's Libraries window so that I can add a dependency on them. I know putting a library in that folder does work because when I found and downloaded a library to fix the dependency on org.eclipse.team.core and put it in the extra\modules directory, NetBeans recognizes it and I see it in the list; therefore I assume that the libraries you suggested I use does not contain all the correct files for NetBeans to include it in the list of libraries. Even if NetBeans were to recognize, it I still don't know if that will satisfy the warning since it is looking for org.apache.ant, not org.apache.tools.ant.

I wonder if I need to put the library in a different location?

Thanks for your help so far. It is much appreciated.
bbergeson
 
Posts: 10
Joined: Thu Feb 04, 2016 5:01 pm

Re: Difficult with dependecies using Python in my RCP app

Postby Jenselme » Fri Sep 02, 2016 6:53 pm

bbergeson wrote:I don't think I was clear in my last statement. When I said that it works in NetBeans it actually still gives me warnings about the missing dependencies but I am able to run in Debug or Run mode without any problems. When I run outside of NetBeans the same warnings cause a pop up and requires me to disable the affected modules or exit. So running outside of NetBeans is doesn't let me ignore the warnings and if I choose to disable the modules and continue then the python plugin doesn't fully work.


OK

bbergeson wrote:Even if NetBeans were to recognize, it I still don't know if that will satisfy the warning since it is looking for org.apache.ant, not org.apache.tools.ant.


I think it will. I just took a quick look at maven and the ant JAR from there is refered as <groupId>org.apache.ant</groupId>. But In the JAR, I see org.apache.tools.ant. Maybe you can try a JAR from there? My previous link from hg.netbeans.org/binaries is old (2014). As for another directory, I'm afraid I don't know. I've never built an RCP application.
Jenselme
 
Posts: 116
Joined: Thu Apr 16, 2015 9:05 pm
Location: Paris, France

Re: Difficult with dependecies using Python in my RCP app

Postby bbergeson » Fri Sep 02, 2016 6:58 pm

I am still trying to get this to work. I had to download and include the following additional libraries, including the right version of some of them, especially wikitext.core:

974EA713463D44F51366BF2E121231BBDAC3A5DC-org.eclipse.mylyn.wikitext.core_2.6.0.v20150901-2143.jar
jsoup-1.7.2.jar
core-3.1.200-v20070522.jar (this is for org.eclipse.ant)
variables-3.2.0-v20070426.jar
app-1.3.200-v20130910-1609.jar
expressions-3.3.0-v20070606-0010.jar
core-3.3.0-I20070607.jar (this is for org.eclipse.team)
org.eclipse.core.resources-3.8.100.v20130521-2026.jar
filesystem-1.1.0-v20070606.jar
runtime-3.5.0.jar
contenttype-3.4.0.jar
jobs-3.4.100.jar
preferences-3.2.300.jar
registry-3.4.100.jar
xercesImpl-2.11.0.jar
com-google-guava.jar
com-sun-jersey.jar
org-joda-time.jar

I am now at the point that I have NO warnings and everything complies fine but still when I run outside of NetBeans I get the following warning:

Warning - could not install some modules:
Python Editor - The module named org.eclipse.mylyn.wikitext.textile.core was needed and not found.
Python Editor - The module named org.eclipse.mylyn.wikitext.confluence.core was needed and not found. Another module could not be installed due to the above problems.

I verified that I have included both of those libraries in the project, AND I do not get any warnings when running from within NetBeans.

So have I some questions for the developers of this plugin.
1. Are you using Maven to build the plugin?
2. What version of NetBeans are you using to develop the plugin?
3. What version of Java are you using to develop the plugin? 32Bit or 64Bit?

This should not be this difficult.
bbergeson
 
Posts: 10
Joined: Thu Feb 04, 2016 5:01 pm

Re: Difficult with dependecies using Python in my RCP app

Postby Jenselme » Fri Sep 02, 2016 8:30 pm

I just did a small experiment: I removed the dependencies on org.eclipse.mylyn.wikitext.textile.core, org.eclipse.mylyn.wikitext.core and org.eclipse.mylyn.wikitext.confluence.core in python.editor (this is the only module requiring them). I don't know what the point of them and I each time I reinstall the Python plugin rg.eclipse.mylyn.wikitext.textile.core, org.eclipse.mylyn.wikitext.core are downloaded and installed again unlike all the other dependencies. This looks strange. Plus, your app doesn't find them. I was able to build and run NetBeans IDE. So I guess we don't really need them. I haven't done much testing thought but the editor seems to work just fine.

I attach the patch to this comment. I think you can try to apply it and see if this works. From my startup log, only org.eclipse.mylyn.wikitext.core is loaded. So you may need to add back to the dependencies list. Or it may just be required by another part of NetBeans IDE. But the other two don't appear anywhere.

To answer you questions:
1. Are you using Maven to build the plugin? No, we use ant. I just quoted maven earlier as a source for JARs.
2. What version of NetBeans are you using to develop the plugin? NetBeans 8.1. I currently build from this commit: 79827caf9c06
3. What version of Java are you using to develop the plugin? 32Bit or 64Bit? Java 7 64 bits (1.7.0_79)

Just in case, you can download my versions of these libraries below. I doubt it will solve the problem but who knows.
Attachments
rm-wikitext-deps-python-editor.patch.txt
Remove deps on wikitext in python.editor
(1.66 KiB) Downloaded 78 times
Jenselme
 
Posts: 116
Joined: Thu Apr 16, 2015 9:05 pm
Location: Paris, France

Re: Difficult with dependecies using Python in my RCP app

Postby bbergeson » Tue Sep 06, 2016 6:28 pm

Thanks for the information. I have never applied a patch before. Based upon the information you provided, I assume that to apply the patch I need to download the source code for the nbpython plugin, apply the changes, and rebuild the plugin?

By the way, when executing ant it said I had to set my JAVA_HOME to a jdk that is 1.8.0_XX, and not 1.7 so how are you building the plugin with 1.7? Also, I used the jar files you provided and they didn't help because they rely on org.apache.ant version 1.7.0 which is not in NetBeans 1.8 and I can't find it.
bbergeson
 
Posts: 10
Joined: Thu Feb 04, 2016 5:01 pm

Re: Difficult with dependecies using Python in my RCP app

Postby Jenselme » Wed Sep 07, 2016 5:10 pm

bbergeson wrote:Thanks for the information. I have never applied a patch before. Based upon the information you provided, I assume that to apply the patch I need to download the source code for the nbpython plugin, apply the changes, and rebuild the plugin?


Yes, you must download the patch and apply it. On linux and the command line, use, in the folder that contains python.editor:
Code: Select all
patch -p1 < /path/to/the/patch


You should also be able to use the apply patch feature of NetBeans. I rarely use it, but if I remember correctly, you need to open python.editor/nbproject/project.xml before applying the patch.

bbergeson wrote:By the way, when executing ant it said I had to set my JAVA_HOME to a jdk that is 1.8.0_XX, and not 1.7 so how are you building the plugin with 1.7?


You may be building from a more recent version of NetBeans than I do. I build it from the commit I gave in my previous post in netbeans-main. I use this script I also have in netbeans/nbbuild/user.build.properties:
Code: Select all
cluster.config=python


This may not be the best way to do it, some ant options could be merged in user.build.properties but it works for me.

bbergeson wrote:Also, I used the jar files you provided and they didn't help because they rely on org.apache.ant version 1.7.0 which is not in NetBeans 1.8 and I can't find it.


Sad to hear that. I guess the proper updated libs exists somewhere. This may also be link to the fact that you rely on a newer version of NetBeans to build your project.
Jenselme
 
Posts: 116
Joined: Thu Apr 16, 2015 9:05 pm
Location: Paris, France

Re: Difficult with dependecies using Python in my RCP app

Postby bbergeson » Thu Sep 08, 2016 4:20 pm

I downloaded the source code for NetBeans and Python and manually edited the project.xml file in the python.editor module according to what I saw in the patch file you sent me. Indeed I don't have to include those extra libraries now, but there were other dependencies that I had to enable which brought me to the following error:

Module Native Execution in ide depends on module com.jcraft.jsch but requests an incompatible specification version.

That does not make sense to me. Why wouldn't two modules that are part of the same NetBeans code base not be compatible?

Regardless this doesn't help me because I can't expose the python plugin in my RCP application because none of the Python modules show up in the list of libraries for me to include in my RCP app. Are you saying that nobody on the nbpython plugin team has tried exposing this plugin in an RCP application? I am not sure what to do next.

Would it be possible for me to get the nbpython plugin source code as a project, not as code place in the contrib directory, and then I can build it myself and create the nbms myself using the NetBeans I have installed rather than the having to download the source code of NetBeans?
bbergeson
 
Posts: 10
Joined: Thu Feb 04, 2016 5:01 pm

Re: Difficult with dependecies using Python in my RCP app

Postby Jenselme » Sun Sep 11, 2016 12:57 pm

bbergeson wrote:I downloaded the source code for NetBeans and Python and manually edited the project.xml file in the python.editor module according to what I saw in the patch file you sent me. Indeed I don't have to include those extra libraries now, but there were other dependencies that I had to enable which brought me to the following error:

Module Native Execution in ide depends on module com.jcraft.jsch but requests an incompatible specification version.

That does not make sense to me. Why wouldn't two modules that are part of the same NetBeans code base not be compatible?


I don't know. I did a quick search in contrib and it looks like no project is using it. In the NetBeans sources, the following projects require it: o.apache.tools.ant.module, php.project, versioning.system.cvss, mobility.deployment.ftpscp, mobility.deployment.ricoh, dlight.nativeexecution It may be more a question for nbusers since I fail to see how it relates to the Python plugin.

bbergeson wrote:Regardless this doesn't help me because I can't expose the python plugin in my RCP application because none of the Python modules show up in the list of libraries for me to include in my RCP app. Are you saying that nobody on the nbpython plugin team has tried exposing this plugin in an RCP application? I am not sure what to do next.


I know I haven't. I don't know if anyone else has tried but I doubt it: the team is quite small and not very active. So I'm affraid I can't help with what to do next.

bbergeson wrote:Would it be possible for me to get the nbpython plugin source code as a project, not as code place in the contrib directory, and then I can build it myself and create the nbms myself using the NetBeans I have installed rather than the having to download the source code of NetBeans?


I don't know. It may be possible but currently the way we build it requires ant and the sources of NetBeans. You may have to change how the build is done to do it. I don't know it this can help you, but I use the script below to build nbms when I need to:

Code: Select all
#!/usr/bin/env bash

if [[ ! -d nbbuild ]]; then
    echo "Launch this in nbbuild dir." >&2
    exit 1
fi

unset JRE_HOME JAVA_BINDIR JAVA_HOME SDK_HOME JDK_HOME JAVA_ROOT
export JAVA_HOME=/usr/java/jdk1.7.0_79
export ANT_HOME=/usr/share/ant/
export ANT_OPTS="-Xmx2048m -XX:MaxPermSize=384m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/antdump.txt -Dcluster.config=python"

rm -rf nbbuild/nbms
ant -Dcluster.config=python build-nonsigned-nbms
Jenselme
 
Posts: 116
Joined: Thu Apr 16, 2015 9:05 pm
Location: Paris, France

Re: Difficult with dependecies using Python in my RCP app

Postby bbergeson » Tue Sep 13, 2016 7:10 pm

I finally have a working solution. To get this working I created a new module in my project and then I copied only the NBPython plugin source files that I needed which ended up being only those from the api, editor, and source modules. I had to depend upon the implementation version of the Editor Options and Code Coverage modules in the IDE module which is not ideal but works. I also had to comment out just a little bit a code in a couple places since I am not actually running python and therefore don't have a PythonPlatform. I modified a few of the bundle.properties files, and moved and updated the layer.xml file. Finally, I compiled the whole thing with java 1.8 update 92 64 bit using source code level of 1.8. I did see a number of warning about the use of deprecated methods or classes. All said and done everything seems to be working great.

Again, thanks for all of your help and clues. Moving forward hopefully this project can be restructured to limit the dependencies to only those that are necessary, and allow a developer to only use the editor portion of the plugin.
bbergeson
 
Posts: 10
Joined: Thu Feb 04, 2016 5:01 pm

Next

Return to How I use NBPython



Who is online

Users browsing this forum: No registered users and 2 guests

cron