Offline CPAN use

Post by nimectos
I looked at "perldoc CPAN" and other info, but I can't seem to
cleanly install CPAN packages offline.

For a starter have a look at CPAN::Mini

http://search.cpan.org/~rjbs/CPAN-Mini/

For a discussion of several people thinking about deeper aspects have
a look at the recent thread in perl-qa:

http://www.nntp.perl.org/group/perl.qa/2011/05/msg12747.html

There you will get an overview of other typical ideas with respective
challenges.

Kind regards,
Steffen

--
Steffen Schwigon <***@renormalist.net>
Dresden Perl Mongers <http://dresden-pm.org/>

(Andreas J. Koenig)

2011-06-26 03:17:25 UTC

Post by nimectos
I looked at "perldoc CPAN" and other info, but I can't seem to
cleanly install CPAN packages offline.
I want to run CPAN normally to install a lot of Perl packages on
one machine (CentOS 5.6 x64 distro just installed, has Perl
5.8.8), capture everything needed to an archive, then take that
archive to another identical fresh machine and install all the
Perl packages WITHOUT any network access.
I fooled with archiving/restoring /root/.cpan/sources, and also
with a bundle, with both methods cpan gave a lot of complaints
about lack of network access, and failed one way ("Can't call
method "color_cmd_tmps" on an undefined value at
/usr/lib/perl5/5.8.8/CPAN.pm line 3784.") or another (said
required package install-0.01 not present - how can this be when
original online install finished successfully?)
What's the best way to do this - whether with just CPAN, and/or
other methods?

The CPAN.pm manpage has this paragraph:

=head2 Floppy, Zip, Offline Mode

CPAN.pm works nicely without network access, too. If you maintain machines
that are not networked at all, you should consider working with C<file:>
URLs. You'll have to collect your modules somewhere first. So
you might use CPAN.pm to put together all you need on a networked
machine. Then copy the $CPAN::Config->{keep_source_where} (but not
$CPAN::Config->{build_dir}) directory on a floppy. This floppy is kind
of a personal CPAN. CPAN.pm on the non-networked machines works nicely
with this floppy. See also below the paragraph about CD-ROM support.

Are you saying you did exactly that and it did not work? I would
consider that a bug. But I need you exact description how you followed
this description. It may be that you misuderstood the process so that
the description needs amending or that the process is broken in the
meantime and needs fixing.

Please let me know how it goes.

--
andreas

nimectos

2011-06-30 06:35:22 UTC

=head2 Floppy, Zip, Offline Mode
CPAN.pm works nicely without network access, too. If you maintain machines
that are not networked at all, you should consider working with C<file:>
URLs. You'll have to collect your modules somewhere first. So
you might use CPAN.pm to put together all you need on a networked
machine. Then copy the $CPAN::Config->{keep_source_where} (but not
$CPAN::Config->{build_dir}) directory on a floppy. This floppy is kind
of a personal CPAN. CPAN.pm on the non-networked machines works nicely
with this floppy. See also below the paragraph about CD-ROM support.
Are you saying you did exactly that and it did not work? I would
consider that a bug. But I need you exact description how you followed
this description. It may be that you misuderstood the process so that
the description needs amending or that the process is broken in the
meantime and needs fixing.
Please let me know how it goes.

I finally succeeded, with some difficulties.

First, the CPAN version (v1.7602) provided with that distro is apparently
broken. I could not install the modules I wanted without encountering at
some point:

Can't call method "color_cmd_tmps" on an undefined value at
/usr/lib/perl5/5.8.8/CPAN.pm line 3784.

BTW, the constant prompting for dependencies bugged me, so I changed
CPAN/Config.pm to:

'prerequisites_policy' => q[follow],

So I tried "cpan Bundle::CPAN" to update CPAN. But although it worked on
the source machine, it wouldn't install offline on the destination.

So instead I then tried (on a fresh machine) "cpan CPAN". That worked.

Then I installed needed modules via "cpan module module ...". That went
fine, but it still prompts:
- MIME::Lite "Add prereqs?" prompt for extra external modules.
- Crypt::SSLeay prompts for install path and live tests.

How can I pre-set the answers to those?

Then it failed, last messages:

...
Result: FAIL
Failed 2/3 test programs. 0/1 subtests failed.
BLHOTSKY/MasonX-Request-WithApacheSession-0.31.tar.gz
2 dependencies missing (HTML::Mason,Apache::Session::Wrapper);
additionally test harness failed
./Build test -- NOT OK
//hint// to see the cpan-testers results for installing this module, try:
reports BLHOTSKY/MasonX-Request-WithApacheSession-0.31.tar.gz
Running Build install
make test had returned bad status, won't install without force

I happened to just try running the same cpan command again, no changes -
and this time it completed OK?! (And got one more set of prompts from
Apache-Session-Wrapper - how preset those?)

So finally everything was installed on the source machine.

So then I grabbed all of the cpan data via, in/as root:

tar -czvf dot-cpan.tar.gz .cpan

Then I went to the clean destination machine (with no network access), and
untar'd the archive. Also, changing to this Config.pm setting seemed to
reduce complaints about being offline:

'connect_to_internet_ok' => q[0],

So I did those same steps again (cpan CPAN, then cpan modules... twice,
getting same prompts, same error), and it did work offline.

However, it takes a long time.

So I was thinking of just archiving the installed modules from the source
machine and unarchiving on the destination, e.g. just tar up:
/usr/lib/perl5/*
/usr/lib64/perl5/*
and restore on the target (ignoring module docs). Would that work? (I'd
still unarchive the .cpan sources so they are present, just wouldn't build
them via cpan commands).

Or is there an alternate approach?

CPAN question: how can I list all installed modules? The closest I could
find is the "r" command, but it's not clear if that is listing every
installed modules, or just those that are candidates for upgrades.

(Andreas J. Koenig)

2011-07-01 01:19:06 UTC

Post by nimectos
I finally succeeded, with some difficulties.
First, the CPAN version (v1.7602) provided with that distro is apparently
broken.

Eight years and more than 120 releases have accumulated on top of this
version. Pity you have to work with such old stuff.

Post by nimectos
[...]
Then I installed needed modules via "cpan module module ...". That went
- MIME::Lite "Add prereqs?" prompt for extra external modules.
- Crypt::SSLeay prompts for install path and live tests.
How can I pre-set the answers to those?

Make sure you have a recent CPAN.pm and read the manpage section

Configuration for individual distributions (Distroprefs)

Post by nimectos
[...]
So I was thinking of just archiving the installed modules from the source
/usr/lib/perl5/*
/usr/lib64/perl5/*
and restore on the target (ignoring module docs). Would that work? (I'd
still unarchive the .cpan sources so they are present, just wouldn't build
them via cpan commands).

If the architectures are sufficiently similar it's likely to work.

Post by nimectos
CPAN question: how can I list all installed modules? The closest I could
find is the "r" command, but it's not clear if that is listing every
installed modules, or just those that are candidates for upgrades.

You're looking for for 'autobundle'. Much to my surprise, 'r' is
undocumented indeed. Will be in the next release. And yes, it only lists
upgradeable modules.

--
andreas

nimectos

2011-07-02 05:57:05 UTC

Post by nimectos
I finally succeeded, with some difficulties.
First, the CPAN version (v1.7602) provided with that distro is apparently
broken.

Eight years and more than 120 releases have accumulated on top of this
version. Pity you have to work with such old stuff.

Thank you for the response.

I need a stable distro, CentOS seems like a good choice, CentOS 6
isn't out quite yet.

Make sure you have a recent CPAN.pm and read the manpage section
Configuration for individual distributions (Distroprefs)

That's good but but seems complex for my needs; I think I'll just
use autoexpect/expect with the cpan command.

Post by nimectos
So I was thinking of just archiving the installed modules from the source
/usr/lib/perl5/*
/usr/lib64/perl5/*
and restore on the target (ignoring module docs). Would that work? (I'd
still unarchive the .cpan sources so they are present, just wouldn't build
them via cpan commands).

If the architectures are sufficiently similar it's likely to work.

Thanks, I'll try it at some point.

You're looking for for 'autobundle'. Much to my surprise, 'r' is
undocumented indeed. Will be in the next release. And yes, it only lists
upgradeable modules.

OK, I'd tried autobundle and seen the output, but thought there
might be a separate command that didn't create the actual bundle
file. I'll just use autobundle and manually clean out the Bundle
directory periodically.

nimectos

2011-12-03 10:23:45 UTC

Post by nimectos
I finally succeeded, with some difficulties.
First, the CPAN version (v1.7602) provided with that distro is apparently
broken.

Eight years and more than 120 releases have accumulated on top of this
version. Pity you have to work with such old stuff.

Thank you for the response.
I need a stable distro, CentOS seems like a good choice, CentOS 6
isn't out quite yet.

Make sure you have a recent CPAN.pm and read the manpage section
Configuration for individual distributions (Distroprefs)

That's good but but seems complex for my needs; I think I'll just
use autoexpect/expect with the cpan command.

If the architectures are sufficiently similar it's likely to work.

Thanks, I'll try it at some point.

You're looking for for 'autobundle'. Much to my surprise, 'r' is
undocumented indeed. Will be in the next release. And yes, it only lists
upgradeable modules.

Following up on my own post here: archiving .cpan, and archiving perl
library directories, works fine for installing the built CPAN and modules
on another [identical] machine, by simply unarchiving those bundles.

Now I am at another phase: a machine built this way needs a few more CPAN
modules installed, and has no network access. How do I install? If I had
network access I would do e.g.:

cpan List::Util List::MoreUtils XML::Simple

So I tried:

cpan Scalar-List-Utils-1.23.tar.gz

which doesn't work; then I tried:

cpan GBARR/Scalar-List-Utils-1.23.tar.gz

which wanted network access and failed.

I am aware of Mini but don't need a mirror - I just need a reasonably
simple way to add a few more Perl modules to a unit that already has many
modules installed via cpan. Perhaps I actually need some tool other than
cpan for this purpose and am just not aware of it.

(Andreas J. Koenig)

2011-12-03 17:13:17 UTC

Post by nimectos
I finally succeeded, with some difficulties.
First, the CPAN version (v1.7602) provided with that distro is apparently
broken.

Eight years and more than 120 releases have accumulated on top of this
version. Pity you have to work with such old stuff.

Thank you for the response.
I need a stable distro, CentOS seems like a good choice, CentOS 6
isn't out quite yet.

Make sure you have a recent CPAN.pm and read the manpage section
Configuration for individual distributions (Distroprefs)

That's good but but seems complex for my needs; I think I'll just
use autoexpect/expect with the cpan command.

If the architectures are sufficiently similar it's likely to work.

Thanks, I'll try it at some point.

You're looking for for 'autobundle'. Much to my surprise, 'r' is
undocumented indeed. Will be in the next release. And yes, it only lists
upgradeable modules.

Following up on my own post here: archiving .cpan, and archiving perl
library directories, works fine for installing the built CPAN and modules
on another [identical] machine, by simply unarchiving those bundles.
Now I am at another phase: a machine built this way needs a few more CPAN
modules installed, and has no network access. How do I install? If I had
cpan List::Util List::MoreUtils XML::Simple
cpan Scalar-List-Utils-1.23.tar.gz
cpan GBARR/Scalar-List-Utils-1.23.tar.gz
which wanted network access and failed.
I am aware of Mini but don't need a mirror - I just need a reasonably
simple way to add a few more Perl modules to a unit that already has many
modules installed via cpan. Perhaps I actually need some tool other than
cpan for this purpose and am just not aware of it.

The section 'Floppy, Zip, Offline Mode' in the CPAN.pm manpage addresses
this. The section is very old predates mini and local::lib and I'm not
sure what the best solution today would be. It makes me grin to find the
word floppy again. Not many people have still the option to use floppies
nowadays:)

You say you do not need a mirror but depending on the way how you
transport the data to the networked machine, a full mirror on a USB
stick might be the most convenient solution.

--
andreas

nimectos

2011-12-04 05:26:34 UTC

Post by nimectos
I finally succeeded, with some difficulties.
First, the CPAN version (v1.7602) provided with that distro is apparently
broken.

Eight years and more than 120 releases have accumulated on top of this
version. Pity you have to work with such old stuff.

Thank you for the response.
I need a stable distro, CentOS seems like a good choice, CentOS 6
isn't out quite yet.

Make sure you have a recent CPAN.pm and read the manpage section
Configuration for individual distributions (Distroprefs)

That's good but but seems complex for my needs; I think I'll just
use autoexpect/expect with the cpan command.

If the architectures are sufficiently similar it's likely to work.

Thanks, I'll try it at some point.

You're looking for for 'autobundle'. Much to my surprise, 'r' is
undocumented indeed. Will be in the next release. And yes, it only lists
upgradeable modules.

Following up on my own post here: archiving .cpan, and archiving perl
library directories, works fine for installing the built CPAN and modules
on another [identical] machine, by simply unarchiving those bundles.
Now I am at another phase: a machine built this way needs a few more CPAN
modules installed, and has no network access. How do I install? If I had
cpan List::Util List::MoreUtils XML::Simple
cpan Scalar-List-Utils-1.23.tar.gz
cpan GBARR/Scalar-List-Utils-1.23.tar.gz
which wanted network access and failed.
I am aware of Mini but don't need a mirror - I just need a reasonably
simple way to add a few more Perl modules to a unit that already has many
modules installed via cpan. Perhaps I actually need some tool other than
cpan for this purpose and am just not aware of it.

The section 'Floppy, Zip, Offline Mode' in the CPAN.pm manpage addresses
this. The section is very old predates mini and local::lib and I'm not
sure what the best solution today would be. It makes me grin to find the
word floppy again. Not many people have still the option to use floppies
nowadays:)
You say you do not need a mirror but depending on the way how you
transport the data to the networked machine, a full mirror on a USB
stick might be the most convenient solution.

Thanks for the response. That man page section seems to assume that you
want the whole CPAN on portable media. But the machine already has a full
CPAN installed - I don't want to replace it, I want to add to it. So it
doesn't make sense to change keep_source_where, I want to point CPAN or
some tool to the module .tar.gz files I want to add, and get them
installed.

Steffen Schwigon

2011-12-05 10:27:57 UTC

I want to point CPAN or some tool to the module .tar.gz files I want
to add, and get them installed.

If you already have the .tgz files,then you just need to unpack them by
yourself and then use “cpan .” inside such a subdir:

cd /tmp/
tar xzf Some-Lib.tar.gz # or -Z/-j for unzip/bunzip2 etc.
cd /tmp/Some-Lib
cpan .

Kind regards,
Steffen

--
Steffen Schwigon <***@renormalist.net>
Dresden Perl Mongers <http://dresden-pm.org/>

nimectos

2011-12-06 06:15:25 UTC

Post by Steffen Schwigon

I want to point CPAN or some tool to the module .tar.gz files I want
to add, and get them installed.

If you already have the .tgz files,then you just need to unpack them by
cd /tmp/
tar xzf Some-Lib.tar.gz # or -Z/-j for unzip/bunzip2 etc.
cd /tmp/Some-Lib
cpan .
Kind regards,
Steffen

Thank you SO much - that's the solution I needed! I did see "cpan ." in
the man CPAN page but it immediately talks about downloading dependencies
so I'd skipped it thinking it was little different than regular cpan use.

I came up with this for my current simple needs of a handful of packages,
with no dependencies I don't already have installed, all are .tar.gz, and
I only need installs, not upgrades:

for MOD_ARCHIVE in *.tar.gz
do
ARCH_CONTENT=`tar -xzvf $MOD_ARCHIVE 2> /dev/null`
[ $? -ne 0 ] && echo "ERROR: Could not extract $MOD_ARCHIVE,
skipping." && continue
ARCH_DIR=`echo "$ARCH_CONTENT" | head -1 | sed -r 's_^([^/]+)/.*_\1_'`
[ $? -ne 0 -o "$ARCH_DIR" == "" ] && echo "ERROR: Could not determine
archive dir for $MOD_ARCHIVE, skipping." && continue
pushd $ARCH_DIR &> /dev/null
MOD_NAME=`grep -hP "^[ \t]+'*NAME'*[ \t]*=>" * 2> /dev/null | sed -r
"s/.*NAME.*[^']+'([^']+)'.*/\1/" | sed -r
"s/.*NAME.*.*q\[([^]]+)\].*/\1/"`
[ $? -ne 0 -o "$MOD_NAME" == "" ] && echo "ERROR: Could not determine
module name for $MOD_ARCHIVE, skipping." && popd &> /dev/null && continue
CHECK_CMD="use $MOD_NAME; exit 0;"
perl -e '$CHECK_CMD' &> /dev/null
if [ $? -eq 0 ]
then
echo "Already installed: $MOD_NAME"
else
echo "Installing: $MOD_NAME from $MOD_ARCHIVE ..."
nice cpan . >> ../cpan-stdout.log 2>>../cpan-stderr.log
[ $? -ne 0 ] && echo "ERROR: 'cpan .' returned error doing
$MOD_NAME, see *.log."
fi
popd &> /dev/null
done

nimectos

2011-12-07 10:27:08 UTC