Hi Gary,
well .. I took a very different way for all this.
Lets start with your questions:
(1) You dont need the *.pyc files to be included. As far as i know, they were be later "generated" on the target machine.
(2) For the dynload you are using a hard (wrong?) way.
First of all, trying to get dynload_shlib.o compile is not the best solution.
the dynload file should be edited/changed like you already noticed/thought.
I removed the other dynload_* files and had a dynload_vxworks.c with this source: http://pastebin.com/4S1Av7K8
You have to use loadModule() under VxWorks to make the lib load.
Since we are running here in User-Space, you have to "link" those functions which are not available in User-Space.
You need a small DKM running which register a sysCallTable with those functions and define them in your RTP aswell.
Here a non-tested example: http://pastebin.com/zkvbrwPP (non-tested since i can't give you in public the whole source since its part of my work .. but if you really need help in this, i could give you a tested example in private).
When you make this run, the import should work without any problem.
I compile my shared lib as a *.pyd and not as an *.so
You have to change the makefile for those lib or change the first link i gave you to load *.so and not '*.pyd.
Now to your question for the possibility to load the shared libs "manually" and forget the dynload-plugin .. well this is a possibility. At first when i didnt make this all work, i compiled the libs with the interpreter in the vxe and added the init-methodes in python/module/config.c
Theoretically you can add all your modules there. The interpreter will be a bit bigger, but well ..
I hope this helped.
If you have more question, dont hesitate.
Best regards.
Gary Kenaley <gkenaley-/***@public.gmane.org> schrieb am 3:34 Dienstag, 18.Februar 2014:
Hi Nachaat,
Thank you for the reply and information! I have 2.7.3 mostly up and running under VxWorks 6.9.3.3 RTP. I have the entire cross build system working smoothly for the executable, parser, shared objects, and the install. I have two remaining questions that I hope you will help me with:
(1) What is the purpose of the *.pyc files, and should I be concerned that the installer does not include them in the final build result? On my native Ubuntu system, the *.pyc files are included, but in the VxWorks build they are not. I am not sure why they would be needed when hundreds of python scripts and shared objects are already available.
(2) Can you tell me how to get the 48 shared objects (*.so) to load automatically when the interpreted needs them (i.e. "import math")?
The specifics of the problems I am having with on the shared objects (dlopen(), etc.) are first that I cannot get python to link with dynload_shlib.o:
Currently I have an python RTP executable that is linked -non-static
with a static library libpython2.7.a, and when it starts it successfully finds libc.so.1 if and only if I have putenv LD_LIBRARY_PATH.
However, to get full python module functionality at run-time, this python vxe needs to
dynamically load-on-demand any of 48 *.so shared libraries that are built AFTER the python RTP executable
is linked -non-static. I believe this means
the 48 shared libraries should be built with the --export-dynamic flag.
All of this works effortlessly in the native Ubuntu build, using the
same source code and very similar Makefile.
When I try to enable dynamic loading [dlopen(), etc.] by building and
linking in dynload_shlib.c (attached) and including it in
libpython2.7.a, the link of the python RTP fails even though there were
no compilation warnings) with:
c++pentium -mrtp -fno-strict-aliasing -D_C99 -D_HAS_C9X -std=c99 -fasm -Wl -MD -MP -non-static -Xbind-lazy -ldl -L/home/demo/WindRiver/vxworks-6.9/target/lib/usr/lib/pentium/PENTIUM4/common -o python Modules/python.o libpython2.7.a
libpython2.7.a(dynload_shlib.o): In function `_PyImport_GetDynLoadFunc':
/romfs/python273/Python/dynload_shlib.c:94: undefined reference to `dlsym'
/romfs/python273/Python/dynload_shlib.c:130: undefined reference to `dlopen'
/romfs/python273/Python/dynload_shlib.c:133: undefined reference to `dlerror'
/romfs/python273/Python/dynload_shlib.c:141: undefined reference to `dlsym'
collect2: ld returned 1 exit status
A slightly more detailed build output for the python RTP is shown about 25 lines below.
I tried dozens of combinations of linker flags, ccpentium, etc., but I always get exactly the same error. Perhaps the source code (dynload_shlib.c) needs to be changed?
I read through the Application Programmer's guide thoroughly and Googled, etc., but there are no examples
showing the build process.
This seems to be the case that the Wind River manual refers to as plugins.
Could you please show me how to make this work? I could hardcode
the dlopenflags for dlopen(pathname, dlopenflags) in dynload_shlib.c if
you would tell me what the correct values should be. Of course it also
needs to link (unless these flags can somehow interfere with that).
I was able to link python with "-Wl,--warn-unresolved-symbols" (with 4 warnings, instead of errors).
Post by nachaat hassisimport math 0xfcfd010 (iPython): RTP 0xfd05010 has been deleted due to signal 11.
Presumably because it tried to blindly call a function that it could not find...
As a possible alternative, now that I have the 48 shared objects built, could I leave the plug-in dynamic loading stubbed out of the libpython2.7.a, and link python.vxe -non-static
against libc.so (as already done) AND the 48 shared objects? The 48
shared objects would be built with "c++pentium -mtune=pentium4 -march=pentium4
-ansi -mrtp -fno-strict-aliasing -D_C99 -D_HAS_C9X -Wall -MD -MP -fpic
-D__SO_PICABIRTP__ -shared -Wl,-shared -D_VX_CPU=_VX_PENTIUM4
-D_VX_TOOL_FAMILY=gnu -D_VX_TOOL=gnu", but without --export-dynamic. Would this work,i.e. would the shared libraries be automatically loadedfrom LD_LIBRARY_PATH whenever the python executable tried to call the functions contained in them?
Thanks,
Gary
More details on the build and link:
***@gamer-linux:/romfs/python273$ make python
ccpentium -c -fno-strict-aliasing -O0 -g -mtune=pentium4 -march=pentium4 -mrtp -fno-strict-aliasing -D_C99 -D_HAS_C9X -std=c99 -fasm -Wall -MD
-MP -D_VX_CPU=_VX_PENTIUM4 -D_VX_TOOL_FAMILY=gnu -D_VX_TOOL=gnu
-I/home/demo/WindRiver/vxworks-6.9/target/usr/h
-I/home/demo/WindRiver/vxworks-6.9/target/usr/h/wrn/coreip -I/romfs/python273/Include -Wno-parentheses -DNDEBUG -fwrapv -O3 -Wall
-Wstrict-prototypes -I. -IInclude -I./Include -O0 -g -mtune=pentium4
-march=pentium4 -ansi -mrtp -fno-strict-aliasing -D_C99 -D_HAS_C9X -Wall -MD -MP -D_VX_CPU=_VX_PENTIUM4 -D_VX_TOOL_FAMILY=gnu -D_VX_TOOL=gnu -I/home/demo/WindRiver/vxworks-6.9/target/usr/h
-I/home/demo/WindRiver/vxworks-6.9/target/usr/h/wrn/coreip
-I/romfs/python273/Include -Wno-parentheses -DPy_BUILD_CORE -o
Python/dynload_shlib.o Python/dynload_shlib.c
ccpentium -c -fno-strict-aliasing -O0 -g -mtune=pentium4 -march=pentium4
-mrtp -fno-strict-aliasing -D_C99 -D_HAS_C9X -std=c99 -fasm -Wall -MD
-MP -D_VX_CPU=_VX_PENTIUM4 -D_VX_TOOL_FAMILY=gnu -D_VX_TOOL=gnu
-I/home/demo/WindRiver/vxworks-6.9/target/usr/h
-I/home/demo/WindRiver/vxworks-6.9/target/usr/h/wrn/coreip
-I/romfs/python273/Include -Wno-parentheses -DNDEBUG -fwrapv -O3 -Wall
-Wstrict-prototypes -I. -IInclude -I./Include -O0 -g -mtune=pentium4
-march=pentium4 -ansi -mrtp -fno-strict-aliasing -D_C99 -D_HAS_C9X -Wall
-MD -MP -D_VX_CPU=_VX_PENTIUM4 -D_VX_TOOL_FAMILY=gnu
-D_VX_TOOL=gnu -I/home/demo/WindRiver/vxworks-6.9/target/usr/h
-I/home/demo/WindRiver/vxworks-6.9/target/usr/h/wrn/coreip
-I/romfs/python273/Include -Wno-parentheses -DPy_BUILD_CORE \
-DSVNVERSION="\"`LC_ALL=C svnversion .`\"" \
-DHGVERSION="\"`LC_ALL=C `\"" \
-DHGTAG="\"`LC_ALL=C `\"" \
-DHGBRANCH="\"`LC_ALL=C `\"" \
-o Modules/getbuildinfo.o ./Modules/getbuildinfo.c
rm -f libpython2.7.a
arpentium rc libpython2.7.a Modules/getbuildinfo.o
arpentium rc libpython2.7.a Parser/acceler.o Parser/grammar1.o
Parser/listnode.o Parser/node.o Parser/parser.o Parser/parsetok.o
Parser/bitset.o Parser/metagrammar.o Parser/firstsets.o Parser/grammar.o
Parser/pgen.o Parser/myreadline.o Parser/tokenizer.o
arpentium rc libpython2.7.a Objects/abstract.o Objects/boolobject.o
Objects/bufferobject.o Objects/bytes_methods.o Objects/bytearrayobject.o
Objects/capsule.o Objects/cellobject.o Objects/classobject.o
Objects/cobject.o Objects/codeobject.o Objects/complexobject.o
Objects/descrobject.o Objects/enumobject.o Objects/exceptions.o
Objects/genobject.o Objects/fileobject.o Objects/floatobject.o
Objects/frameobject.o Objects/funcobject.o Objects/intobject.o
Objects/iterobject.o Objects/listobject.o Objects/longobject.o
Objects/dictobject.o
Objects/memoryobject.o Objects/methodobject.o Objects/moduleobject.o
Objects/object.o Objects/obmalloc.o Objects/rangeobject.o
Objects/setobject.o Objects/sliceobject.o Objects/stringobject.o
Objects/structseq.o Objects/tupleobject.o Objects/typeobject.o
Objects/weakrefobject.o
Objects/unicodeobject.o Objects/unicodectype.o
arpentium rc libpython2.7.a Python/_warnings.o Python/Python-ast.o
Python/asdl.o Python/ast.o Python/bltinmodule.o Python/ceval.o
Python/compile.o Python/codecs.o Python/errors.o Python/frozen.o
Python/frozenmain.o Python/future.o Python/getargs.o
Python/getcompiler.o
Python/getcopyright.o Python/getplatform.o Python/getversion.o
Python/graminit.o Python/import.o Python/importdl.o Python/marshal.o
Python/modsupport.o Python/mystrtoul.o Python/mysnprintf.o
Python/peephole.o Python/pyarena.o Python/pyctype.o Python/pyfpe.o
Python/pymath.o Python/pystate.o Python/pythonrun.o Python/random.o
Python/structmember.o Python/symtable.o Python/sysmodule.o
Python/traceback.o Python/getopt.o Python/pystrcmp.o Python/pystrtod.o
Python/dtoa.o Python/formatter_unicode.o Python/formatter_string.o Python/dynload_shlib.o Python/thread.o
arpentium rc libpython2.7.a Modules/config.o Modules/getpath.o Modules/main.o Modules/gcmodule.o
arpentium rc libpython2.7.a Modules/threadmodule.o
Modules/signalmodule.o Modules/posixmodule.o Modules/errnomodule.o
Modules/_sre.o Modules/_codecsmodule.o Modules/_weakref.o
Modules/zipimport.o Modules/symtablemodule.o Modules/xxsubtype.o
ranlibpentium libpython2.7.a
ccpentium -mrtp -fno-strict-aliasing -D_C99 -D_HAS_C9X -std=c99 -fasm
-Wall -MD -MP
-L/home/demo/WindRiver/vxworks-6.9/target/lib/usr/lib/pentium/PENTIUM4/common
-mrtp -fno-strict-aliasing -D_C99 -D_HAS_C9X -std=c99 -fasm -Wall -MD
-MP -non-static -o python
Modules/python.o libpython2.7.a
libpython2.7.a(dynload_shlib.o): In function `_PyImport_GetDynLoadFunc':
/romfs/python273/Python/dynload_shlib.c:94: undefined reference to `dlsym'
/romfs/python273/Python/dynload_shlib.c:130: undefined reference to `dlopen'
/romfs/python273/Python/dynload_shlib.c:133: undefined reference to `dlerror'
/romfs/python273/Python/dynload_shlib.c:141: undefined reference to `dlsym'
collect2: ld returned 1 exit status
make: *** [python] Error 1
On Monday, February 17, 2014 12:29 AM, nachaat hassis <nachaat05-***@public.gmane.org> wrote:
Hi Gary,
I was trying to port Python 3.0 at first .. after porting it, i noticed that Twisted and Buildbot support only Phyton 2.7
So i had to port Python 2.7 ..
I was working with VxWorks 6.9 as well (ARM)..
well, after trying to port the Python as a DKM, i had it working but i had many problems with Twisted and Buildbot .. (they use many signals, pipes, ..) many POSIX stuff ..
So i took another direction and tried to make it run as an RTP.
With the RTP-Version i had a better result. POSIX stuff is working, my buildbot and twisted are working (with few changes since fork and exec are not supported for example) ..
But i made it work .. I have the feeling that the Python-Interpreter was more stable as a DKM but maybe its just an impression ..
I had the loadable shared objects part done aswell .. If you need help with this part, just ask ..
and thanks for the replay.
Best regards ..
Post by nachaat hassisHi Everybody,
im trying to port Python on VxWorks or to find binarys for VxWorks.
My search on the internet was not very successfull till now.
In this
page http://docstore.mik.ua/orelly/other/python/0596001886_pythonian-chp-2-sect-2.html you
Post by nachaat hassisThe site http://www.python.org/download/ provides links to binary
distributions for Macintosh,
Post by nachaat hassisOS/2, Amiga, RISC OS, QNX, VxWorks, IBM AS/400, Sony PlayStation 2, and
Sharp Zaurus.
Post by nachaat hassisbut i cant find any binarys for VxWorks there.
Is it even possible to port Python on VxWorks? lets say 2.7 ..
The version is not very important to me .. i just need someone to point me
to the right direction ..
Post by nachaat hassisAn old binarys of Python for VxWorks would do it..
Did someone hear that anybody ported Python on Vxworks?
My goal is to make BuildBot run on VxWorks
..
Nachaat,
I am currently working on a Python 2.7
for VxWorks 6.9.3 x86 user space.
The best resource is: http://wiki.osdev.org/Porting_Python
I am using User Space Real Time Processes (RTPs) because they have POSIX
PSE52+ functionality and full memory protection.
I had the basic 2.6.1 interpreter running this way, but it's fairly useless
without the loadable shared objects (the osdev site should help a lot with
those cross build aspects).
If I get it working satisfactorily, I will post back.
-Gary