Hi Victor, On 05/17/10 09:07, pitillo@crux-arm.nu wrote:

* Victor Martinez (pitillo@crux-arm.nu) wrote:

Hello,
I have been doing some test with native compilations under handhelds (at 
the moment I'm trying on htc-prophet).

nice, I tried on my jornada too but nothing serious, just for fun ;D

I noticed what Fredick told me about the -pipe flag, which seems to kill 
memory (only with 64Mb).
I want to know your opinions about this and another topic related to native 
compilations and the commits in native git repos.
I think it can be a good idea to remove the -pipe flags in ports which have 
it hardcoded (like perl). In this way we can let native building work with 
devices which don't have too much amount of ram.

well since pda's are not designed for native compilations IMHO a comment in a doc would be enough. Anyways we should try to keep ports as transparent as possible and we should let the user the freedom to use his own flags, so I like more the idea of replace flags instead of remove some flags (all ports must accept user defined flags in pkgmk.conf whenever be possible)

We can see this errors reported at the end of the pkgbuild log:
"cc: Internal error: Killed (program cc1)
Please submit a full bug report.

Related to ram are sources too, xz and lzma needs more ram to be 
uncompressed (their size is smaller that bz2 sources) and then it's 
impossible in devices with small amount of ram.

A full report about xz/lzma problem:
lzma:
=======> Building '/nfs/crux/pkg/curl#7.20.1-1.pkg.tar.gz'.
bsdtar -p -o -C /nfs/crux/work/src -xf /nfs/crux/src/curl-7.20.1.tar.lzma
unlzma: (stdin): Memory usage limit reached
unlzma: Limit was 22 MiB, but 33 MiB would have been needed
bsdtar: Unrecognized archive format
bsdtar: Child process exited with status 1
bsdtar: Error exit delayed from previous errors.
=======> ERROR: Building '/nfs/crux/pkg/curl#7.20.1-1.pkg.tar.gz' failed.
xz:
bsdtar -p -o -C /nfs/crux/work/src -xf /nfs/crux/src/coreutils-8.5.tar.xz
unxz: (stdin): Memory usage limit reached
unxz: Limit was 22 MiB, but 65 MiB would have been needed
bsdtar: Unrecognized archive format
bsdtar: Child process exited with status 1
bsdtar: Error exit delayed from previous errors.
=======> ERROR: Building '/nfs/crux/pkg/coreutils#8.5-1.pkg.tar.gz' failed.

Using gz/bz2 sources avoid this problem, but we can get the problem exposed above.

same on my jornada 720, but only 11MiB:[...]=======> Building '/usr/pkg/diffutils#3.0-1.pkg.tar.gz'. bsdtar -p -o -C /dev/shm/diffutils-work/src -xf /usr/pkg/src/diffutils-3.0.tar.xz unxz: (stdin): Memory usage limit reached unxz: Limit was 11 MiB, but 65 MiB would have been needed [...]well I found something interesting in the code (see Changelog line) 11MB is around 30% of 32MB (which has the j720 device) and 22MB is around 30% of 64MB (your htc-prophet device) [...]$ cd /dev/shm/xz-work/src/xz-4.999.9beta $ grep 'Memory usage limit' -r * ChangeLog: - Memory usage limit is now about 30 % for uncompression po/xz.pot:msgid "Memory usage limit (%<PRIu64> MiB) is too small for the given filter setup (%<PRIu64> MiB)" po/xz.pot:msgid "Memory usage limit reached" src/liblzma/common/alone_decoder.c: /// Memory usage limit src/liblzma/common/index_decoder.c: /// Memory usage limit src/liblzma/common/stream_decoder.c: /// Memory usage limit src/liblzma/api/lzma/base.h: * \brief Memory usage limit was reached src/liblzma/api/lzma/container.h: * - LZMA_MEMLIMIT_ERROR: Memory usage limit was reached. src/liblzma/api/lzma/index.h: * - LZMA_MEMLIMIT_ERROR: Memory usage limit was reached. src/liblzma/api/lzma/lzma.h: * - Memory usage limit src/xz/coder.c: message_fatal(_("Memory usage limit (%" PRIu64 " MiB) is too small " src/xz/hardware.c:/// Memory usage limit src/xz/message.c: return _("Memory usage limit reached"); src/xzdec/xzdec.c: msg = "Memory usage limit reached"; [...] and this is the changeset which introduced it: [...] commit 2672bcc9f85ba28ff648e092e9eb4cd9e69ce418 Author: Lasse Collin <lasse.collin@tukaani.org> Date: Sun Mar 7 13:29:28 2010 +0200 Increase the default memory usage limit on "low-memory" systems. Previously the default limit was always 40 % of RAM. The new limit is a little bit more complex: - If 40 % of RAM is at least 80 MiB, 40 % of RAM is used as the limit. - If 80 % of RAM is over 80 MiB, 80 MiB is used as the limit. - Otherwise 80 % of RAM is used as the limit. This should make it possible to decompress files created with "xz -9" on more systems. Swapping is generally more expected on systems with less RAM, so higher default limit on them shouldn't cause too bad surprises in terms of heavy swapping. Instead, the higher default limit should reduce the number of bad surprises when it used to prevent decompression of files created with "xz -9". The DoS prevention system shouldn't be a DoS itself. Note that even with the new default limit, a system with 64 MiB RAM cannot decompress files created with "xz -9" without user overriding the limit. This should be OK, because if xz is going to need more memory than the system has RAM, it will run very very slowly and thus it's good that user has to override the limit in that case. [...]

I hope this can be interesting. Waiting your comments and suggestions.

maybe we could try to find a solution/patch (maybe I'll play at night), talk with xz developers about that, or as a last resorttry an alternative like you said (tar.bz2 instead of xz sources)

Regards, Victor.

Best regards,

-- 
Jose V Beneyto | http://sepen.mine.nu/