Search:  
Gentoo Wiki

TIP_Speed_up_portage_with_Psyco


Psyco is a Python extension which speeds up Python code by compiling it, and therefore accelerates Portage by a factor of 2 to 5.


Warning: Using Psyco may cause stability issues with portage. Using Psyco is not recommended by the Gentoo Portage developers. Use at your own risk. Bugs filed where Psyco is used will be marked INVALID. Please attempt to reproduce the problem with Psyco disabled. Psyco apparently will also never support non-32-bit platforms (e.g., Athlon64) so don't bother unmasking it for those (see "Only_32-bit" for more details below).


Installation

First emerge psyco:

emerge dev-python/psyco


Next, you'll need the following wrapper program:

File: pemerge
#!/usr/bin/python -O

# copyright (C) 2006 Eugene St Leger (GrimRC)
# licensed to you under terms of GPL v2
# modified by Jordan Callicoat (MonkeeSage)

# psyco wrapper-script - version 6

import os, sys, re, imp

# strip the 'p' prefix from the name of this file to
# use as the name of the file to invoke
name = os.path.basename(__file__)[1:]

path = os.getcwd()
filename = None

# check if file to invoke resides in same directory
# if not search the PATH for it
if name in os.listdir(path):
	filename = os.path.join(path, name)
else:
	for entry in os.getenv('PATH').split(':'):
		if os.path.isdir(entry) and name in os.listdir(entry):
			filename = os.path.join(entry, name)

if not filename:
	sys.exit("File not found: %s" % name)

try:
	import psyco
	sys.stdout.write("psyco imported")
except ImportError:
	sys.stderr.write("psyco failed to import\n")
	pass

if __name__ == "__main__":
	try:
		psyco.full()
		sys.stdout.write(" & activated\n")
	except:
		sys.stdout.write("\n")
		pass

	# use imp to import & set __name__ = '__main__'
	try:
		fileobj = open(filename, 'U')
		imp.load_source('__main__', path, fileobj)
	finally:
		# try to close fileobj if it is still open (seems sometimes necessary and sometimes not)
		try:
			fileobj.close()
		except:
			pass


Copy this text to a file named pemerge somewhere in the PATH, and make it executable (chmod a+x pemerge), then use pemerge rather than emerge to get an accelerated emerge. To disable acceleration, simply call emerge as you normally would rather than calling pemerge. The same procedure can be done for any python program; simply make a copy of pemerge and change the filename to whatever the name of the other program is (e.g., psonata).

Note: You can actually use any single-letter prefix, you don't have to use "p"--you could name the file "aemerge" for example.

Usage of psyco.full() is faster than psyco.profile(); psyco.profile() only compiles a few functions on load so the compiling won't take up too much time. psyco.full() will always compile everything, so initial loading is slower but the program speed is faster.

Only_32-bit

On Saturday, February 2nd, 2008, where `uname -a` reported:

Linux Ryu 2.6.23-gentoo-r3 #1 SMP Mon Jan 21 17:23:13 UTC 2008 x86_64 AMD Athlon(tm) 64 X2 Dual Core Processor 4200+ AuthenticAMD GNU/Linux

an attempt to unmask && `emerge dev-python/psyco` yielded:

File: error
...
c/codegen.h:15:3: error: #error "-----------------------------------------------------"
c/codegen.h:16:3: error: #error "Sorry, non-32-bit platforms are not supported at all."
c/codegen.h:17:3: error: #error "You may try with a Python compiled in 32-bit         "
c/codegen.h:18:3: error: #error "compatibility mode.  Note that Psyco will probably   "
c/codegen.h:19:3: error: #error "never support non-32-bit platforms, as it is no      "
c/codegen.h:20:3: error: #error "longer actively developed.  Instead, the PyPy group  "
c/codegen.h:21:3: error: #error "plans to replace it with a more flexible and easily  "
c/codegen.h:22:3: error: #error "retargettable Psyco-for-PyPy during the year 2006.   "
c/codegen.h:23:3: error: #error "See http://codespeak.net/pypy/                       "
c/codegen.h:24:3: error: #error "-----------------------------------------------------"

Retrieved from "http://www.gentoo-wiki.info/TIP_Speed_up_portage_with_Psyco"

Last modified: Fri, 14 Mar 2008 23:03:00 +0000 Hits: 26,200