source: trunk/essentials/dev-lang/perl/pod/perl5005delta.pod

=head1 NAME

perl5005delta - what's new for perl5.005

=head1 DESCRIPTION

This document describes differences between the 5.004 release and this one.

=head1 About the new versioning system

Perl is now developed on two tracks: a maintenance track that makes
small, safe updates to released production versions with emphasis on
compatibility; and a development track that pursues more aggressive
evolution.  Maintenance releases (which should be considered production
quality) have subversion numbers that run from C<1> to C<49>, and
development releases (which should be considered "alpha" quality) run
from C<50> to C<99>.

Perl 5.005 is the combined product of the new dual-track development
scheme.

=head1 Incompatible Changes

=head2 WARNING:  This version is not binary compatible with Perl 5.004.

Starting with Perl 5.004_50 there were many deep and far-reaching changes
to the language internals.  If you have dynamically loaded extensions
that you built under perl 5.003 or 5.004, you can continue to use them
with 5.004, but you will need to rebuild and reinstall those extensions
to use them 5.005.  See F<INSTALL> for detailed instructions on how to
upgrade.

=head2 Default installation structure has changed

The new Configure defaults are designed to allow a smooth upgrade from
5.004 to 5.005, but you should read F<INSTALL> for a detailed
discussion of the changes in order to adapt them to your system.

=head2 Perl Source Compatibility

When none of the experimental features are enabled, there should be
very few user-visible Perl source compatibility issues.

If threads are enabled, then some caveats apply. C<@_> and C<$_> become
lexical variables.  The effect of this should be largely transparent to
the user, but there are some boundary conditions under which user will
need to be aware of the issues.  For example, C<local(@_)> results in
a "Can't localize lexical variable @_ ..." message.  This may be enabled
in a future version.

Some new keywords have been introduced.  These are generally expected to
have very little impact on compatibility.  See L<New C<INIT> keyword>,
L<New C<lock> keyword>, and L<New C<qrE<sol>E<sol>> operator>.

Certain barewords are now reserved.  Use of these will provoke a warning
if you have asked for them with the C<-w> switch.
See L<C<our> is now a reserved word>.

=head2 C Source Compatibility

There have been a large number of changes in the internals to support
the new features in this release.

=over 4

=item *

Core sources now require ANSI C compiler

An ANSI C compiler is now B<required> to build perl.  See F<INSTALL>.

=item *

All Perl global variables must now be referenced with an explicit prefix

All Perl global variables that are visible for use by extensions now
have a C<PL_> prefix.  New extensions should C<not> refer to perl globals
by their unqualified names.  To preserve sanity, we provide limited
backward compatibility for globals that are being widely used like
C<sv_undef> and C<na> (which should now be written as C<PL_sv_undef>,
C<PL_na> etc.)

If you find that your XS extension does not compile anymore because a
perl global is not visible, try adding a C<PL_> prefix to the global
and rebuild.

It is strongly recommended that all functions in the Perl API that don't
begin with C<perl> be referenced with a C<Perl_> prefix.  The bare function
names without the C<Perl_> prefix are supported with macros, but this
support may cease in a future release.

See L<perlapi>.

=item *

Enabling threads has source compatibility issues

Perl built with threading enabled requires extensions to use the new
C<dTHR> macro to initialize the handle to access per-thread data.
If you see a compiler error that talks about the variable C<thr> not
being declared (when building a module that has XS code),  you need
to add C<dTHR;> at the beginning of the block that elicited the error.

The API function C<perl_get_sv("@",FALSE)> should be used instead of
directly accessing perl globals as C<GvSV(errgv)>.  The API call is
backward compatible with existing perls and provides source compatibility
with threading is enabled.

See L<"C Source Compatibility"> for more information.

=back

=head2 Binary Compatibility

This version is NOT binary compatible with older versions.  All extensions
will need to be recompiled.  Further binaries built with threads enabled
are incompatible with binaries built without.  This should largely be
transparent to the user, as all binary incompatible configurations have
their own unique architecture name, and extension binaries get installed at
unique locations.  This allows coexistence of several configurations in
the same directory hierarchy.  See F<INSTALL>.

=head2 Security fixes may affect compatibility

A few taint leaks and taint omissions have been corrected.  This may lead
to "failure" of scripts that used to work with older versions.  Compiling
with -DINCOMPLETE_TAINTS provides a perl with minimal amounts of changes
to the tainting behavior.  But note that the resulting perl will have
known insecurities.

Oneliners with the C<-e> switch do not create temporary files anymore.

=head2 Relaxed new mandatory warnings introduced in 5.004

Many new warnings that were introduced in 5.004 have been made
optional.  Some of these warnings are still present, but perl's new
features make them less often a problem.  See L<New Diagnostics>.

=head2 Licensing

Perl has a new Social Contract for contributors.  See F<Porting/Contract>.

The license included in much of the Perl documentation has changed.
Most of the Perl documentation was previously under the implicit GNU
General Public License or the Artistic License (at the user's choice).
Now much of the documentation unambiguously states the terms under which
it may be distributed.  Those terms are in general much less restrictive
than the GNU GPL.  See L<perl> and the individual perl manpages listed
therein.

=head1 Core Changes


=head2 Threads

WARNING: Threading is considered an B<experimental> feature.  Details of the
implementation may change without notice.  There are known limitations
and some bugs.  These are expected to be fixed in future versions.

See F<README.threads>.

=head2 Compiler

WARNING: The Compiler and related tools are considered B<experimental>.
Features may change without notice, and there are known limitations
and bugs.  Since the compiler is fully external to perl, the default
configuration will build and install it.

The Compiler produces three different types of transformations of a
perl program.  The C backend generates C code that captures perl's state
just before execution begins.  It eliminates the compile-time overheads
of the regular perl interpreter, but the run-time performance remains
comparatively the same.  The CC backend generates optimized C code
equivalent to the code path at run-time.  The CC backend has greater
potential for big optimizations, but only a few optimizations are
implemented currently.  The Bytecode backend generates a platform
independent bytecode representation of the interpreter's state
just before execution.  Thus, the Bytecode back end also eliminates
much of the compilation overhead of the interpreter.

The compiler comes with several valuable utilities.

C<B::Lint> is an experimental module to detect and warn about suspicious
code, especially the cases that the C<-w> switch does not detect.

C<B::Deparse> can be used to demystify perl code, and understand
how perl optimizes certain constructs.

C<B::Xref> generates cross reference reports of all definition and use
of variables, subroutines and formats in a program.

C<B::Showlex> show the lexical variables used by a subroutine or file
at a glance.

C<perlcc> is a simple frontend for compiling perl.

See C<ext/B/README>, L<B>, and the respective compiler modules.

=head2 Regular Expressions

Perl's regular expression engine has been seriously overhauled, and
many new constructs are supported.  Several bugs have been fixed.

Here is an itemized summary:

=over 4

=item Many new and improved optimizations

Changes in the RE engine:

        Unneeded nodes removed;
        Substrings merged together;
        New types of nodes to process (SUBEXPR)* and similar expressions
            quickly, used if the SUBEXPR has no side effects and matches
            strings of the same length;
        Better optimizations by lookup for constant substrings;
        Better search for constants substrings anchored by $ ;

Changes in Perl code using RE engine:

        More optimizations to s/longer/short/;
        study() was not working;
        /blah/ may be optimized to an analogue of index() if $& $` $' not seen;
        Unneeded copying of matched-against string removed;
        Only matched part of the string is copying if $` $' were not seen;

=item Many bug fixes

Note that only the major bug fixes are listed here.  See F<Changes> for others.

        Backtracking might not restore start of $3.
        No feedback if max count for * or + on "complex" subexpression
            was reached, similarly (but at compile time) for {3,34567}
        Primitive restrictions on max count introduced to decrease a 
            possibility of a segfault;
        (ZERO-LENGTH)* could segfault;
        (ZERO-LENGTH)* was prohibited;
        Long REs were not allowed;
        /RE/g could skip matches at the same position after a 
          zero-length match;

=item New regular expression constructs

The following new syntax elements are supported:

        (?<=RE)
        (?<!RE)
        (?{ CODE })
        (?i-x)
        (?i:RE)
        (?(COND)YES_RE|NO_RE)
        (?>RE)
        \z

=item New operator for precompiled regular expressions

See L<New C<qrE<sol>E<sol>> operator>.

=item Other improvements

        Better debugging output (possibly with colors),
            even from non-debugging Perl;
        RE engine code now looks like C, not like assembler;
        Behaviour of RE modifiable by `use re' directive;
        Improved documentation;
        Test suite significantly extended;
        Syntax [:^upper:] etc., reserved inside character classes;

=item Incompatible changes

        (?i) localized inside enclosing group;
        $( is not interpolated into RE any more;
        /RE/g may match at the same position (with non-zero length)
            after a zero-length match (bug fix).

=back

See L<perlre> and L<perlop>.

=head2   Improved malloc()

See banner at the beginning of C<malloc.c> for details.

=head2 Quicksort is internally implemented

Perl now contains its own highly optimized qsort() routine.  The new qsort()
is resistant to inconsistent comparison functions, so Perl's C<sort()> will
not provoke coredumps any more when given poorly written sort subroutines.
(Some C library C<qsort()>s that were being used before used to have this
problem.)  In our testing, the new C<qsort()> required the minimal number
of pair-wise compares on average, among all known C<qsort()> implementations.

See C<perlfunc/sort>.

=head2 Reliable signals

Perl's signal handling is susceptible to random crashes, because signals
arrive asynchronously, and the Perl runtime is not reentrant at arbitrary
times.

However, one experimental implementation of reliable signals is available
when threads are enabled.  See C<Thread::Signal>.  Also see F<INSTALL> for
how to build a Perl capable of threads.

=head2 Reliable stack pointers

The internals now reallocate the perl stack only at predictable times.
In particular, magic calls never trigger reallocations of the stack,
because all reentrancy of the runtime is handled using a "stack of stacks".
This should improve reliability of cached stack pointers in the internals
and in XSUBs.

=head2 More generous treatment of carriage returns

Perl used to complain if it encountered literal carriage returns in
scripts.  Now they are mostly treated like whitespace within program text.
Inside string literals and here documents, literal carriage returns are
ignored if they occur paired with linefeeds, or get interpreted as whitespace
if they stand alone.  This behavior means that literal carriage returns
in files should be avoided.  You can get the older, more compatible (but
less generous) behavior by defining the preprocessor symbol
C<PERL_STRICT_CR> when building perl.  Of course, all this has nothing
whatever to do with how escapes like C<\r> are handled within strings.

Note that this doesn't somehow magically allow you to keep all text files
in DOS format.  The generous treatment only applies to files that perl
itself parses.  If your C compiler doesn't allow carriage returns in
files, you may still be unable to build modules that need a C compiler.

=head2 Memory leaks

C<substr>, C<pos> and C<vec> don't leak memory anymore when used in lvalue
context.  Many small leaks that impacted applications that embed multiple
interpreters have been fixed.

=head2 Better support for multiple interpreters

The build-time option C<-DMULTIPLICITY> has had many of the details
reworked.  Some previously global variables that should have been
per-interpreter now are.  With care, this allows interpreters to call
each other.  See the C<PerlInterp> extension on CPAN.

=head2 Behavior of local() on array and hash elements is now well-defined

See L<perlsub/"Temporary Values via local()">.

=head2 C<%!> is transparently tied to the L<Errno> module

See L<perlvar>, and L<Errno>.

=head2 Pseudo-hashes are supported

See L<perlref>.

=head2 C<EXPR foreach EXPR> is supported

See L<perlsyn>.

=head2 Keywords can be globally overridden

See L<perlsub>.

=head2 C<$^E> is meaningful on Win32

See L<perlvar>.

=head2 C<foreach (1..1000000)> optimized

C<foreach (1..1000000)> is now optimized into a counting loop.  It does
not try to allocate a 1000000-size list anymore.

=head2 C<Foo::> can be used as implicitly quoted package name

Barewords caused unintuitive behavior when a subroutine with the same
name as a package happened to be defined.  Thus, C<new Foo @args>,
use the result of the call to C<Foo()> instead of C<Foo> being treated
as a literal.  The recommended way to write barewords in the indirect
object slot is C<new Foo:: @args>.  Note that the method C<new()> is
called with a first argument of C<Foo>, not C<Foo::> when you do that.

=head2 C<exists $Foo::{Bar::}> tests existence of a package

It was impossible to test for the existence of a package without
actually creating it before.  Now C<exists $Foo::{Bar::}> can be
used to test if the C<Foo::Bar> namespace has been created.

=head2 Better locale support

See L<perllocale>.

=head2 Experimental support for 64-bit platforms

Perl5 has always had 64-bit support on systems with 64-bit longs.
Starting with 5.005, the beginnings of experimental support for systems
with 32-bit long and 64-bit 'long long' integers has been added.
If you add -DUSE_LONG_LONG to your ccflags in config.sh (or manually
define it in perl.h) then perl will be built with 'long long' support.
There will be many compiler warnings, and the resultant perl may not
work on all systems.  There are many other issues related to
third-party extensions and libraries.  This option exists to allow
people to work on those issues.

=head2 prototype() returns useful results on builtins