This is a long reply rather than send a bunch of shorter emails.
> On Jun 27, 2024, at 2:10 PM, Deleu <[email protected]> wrote:
>
> Overall, I think PHP has already reached the limit of surviving with only PSR-4 and Composer.
> Single class files were a great solution to get us out of the nightmare of require and
> import on top of PHP files. But more than once I have had the desire to declare a
> couple of interfaces in a single file, or a handful of Enums, etc.
This.
I cannot overemphasize how nice it is to work in Go where I can put almost any code I want in any
file I want without having to think about autoloading.
It is great when writing proofs of concept and having all the code in one place makes it easier to
reason about. Once fleshed out you can then organize into multiple files, but still get to keep
highly related code in the same files.
> On Jun 27, 2024, at 3:02 PM, Michael Morris <[email protected]> wrote:
> Thanks. The sticking point is what degree of change should be occurring. PHP isn't as
> behind an 8-ball as JavaScript is since the dev can choose their PHP version and hence deprecation
> works most of the time for getting rid of old stuff. But not always. Changes that are incompatible
> with what came before need a way to do things the old way during transition.
As I understand the proposal, this would have no BC issues for code not in modules. PHP could then
set rules for code in modules that would not to be directly compatible with code outside modules.
That's how it works in JavaScript, at least as I have experienced, and I'd say it works
pretty well.
> Again, see PHP 6 and unicode, which snowballed until it was clear that even if PHP 6 had been
> completed it wouldn't be able to run most PHP 5 code.
At least to me this does not feel as big as trying to implement unicode.
> 2. No need for autoloaders with modules; I assume this would be obvious, right?
>
> Depends largely on whether modules can include and require to get access to old code. I also
> didn't discuss how they behave - do they share their variables with includes and requires?
I was presuming that all old code would use autoloaders but modules would be free to do it a better
way.
If you need to call code from a namespace from inside a module, sure, the autoloader would be
needed.
> 6. Modules should be directories, not .php files. Having each file be a module makes code org
> really hard.
>
> Yes, but that is how JavaScript currently handles things. It is currently necessary when making
> large packages to have an index.js that exports out the public members of the module. This entry
> point is configurable through the package.json of the module.
I am envisioning that there could be a module metadata file that would have everything that PHP
needs to handle the module. It could even be binary, using protobufs:
https://github.com/protobuf-c/protobuf-c
The php CLI could have an option to generate this file making it easy for IDEs to generate the file,
or generic file watchers to generate. This would mean that within a module there would be no need
for an autoloader.
If the module metadata file does n0t exist, PHP could generate it on the fly. If the file is
obviously out-of-date given a new file, PHP could re-generate. If PHP can't write the file,
such as on a production server, it throws a warning and regenerates for in-memory use each page
load.
It iss also possible that instead of protobuf the module file could actually be a phar file, or the
equivalent of a phar file optimized to allow PHP to load, access and execute that code as fast as
possible.
>
> 7. Modules would have a symbol table metadata file generated by IDEs and during deployment.
>
> Node.js uses package.json and the attendant npm to do this sort of prep work. And it's a
> critical part of this since modules can be versioned, and different modules may need to run
> different specific versions of other modules.
node_modules IMO is one of the worse things about the JavaScript ecosystem. Who has not seen the
meme about node_modules being worse than a black hole?
I would argue that PHP itself not be involved in trying to manage versions. Let Composer do that, or
whatever other tool developers currently use to manage versions, or new tools developed later.
> 9. .php files in modules as identified by metadata file should not be loadable via HTTP(S).
>
> Those are implementation details a little further down the road than we're ready for, I
> think.
But ensuring that it is possible to disallow loading needs to be contemplated in the design. PHP has
to be able to know what is a module and what isn't without expensive processes.
> 10. Having exports separate from functions and classes seems like it would be problematic.
>
> Again, this is how they work in JavaScript. Not saying that's the best approach, but even
> if problematic it's a solved problem.
I have evidently not written enough JavaScript to realize that.
> I'm also interested in learning on how other module systems out there do work.
I am very familiar with modules (packages) in GoLang and think PHP could benefit from considering
how they work, too.
> On Jun 27, 2024, at 3:22 PM, Michael Morris <[email protected]> wrote:
> Composer would need a massive rewrite to be a part of this since it currently requires the file
> once it determines it should do so. If we do a system where import causes the parser to act
> differently then that alone means imports can't be dealt with in the same manner as other
> autoloads.
That is why I am strongly recommending a modern symbol resolution system within modules vs.
autoloading.
>> I'm not fond of this either.
>
> There will need to be a way to define the entrypoint php. I think index.php is reasonable, and
> if another entry point is desired it can be called out -> "mypackage/myentry.php"
Why is an entry point needed? If there is a module metadata file as I am proposing PHP can get all
the information it needs from that file. Maybe that is the .phm file?
> On Jun 27, 2024, at 4:54 PM, Rob Landers <[email protected]> wrote:
>> Thanks. The sticking point is what degree of change should be occurring. PHP isn't as
>> behind an 8-ball as JavaScript is since the dev can choose their PHP version and hence deprecation
>> works most of the time for getting rid of old stuff. But not always. Changes that are incompatible
>> with what came before need a way to do things the old way during transition. Again, see PHP 6 and
>> unicode, which snowballed until it was clear that even if PHP 6 had been completed it wouldn't
>> be able to run most PHP 5 code.
>
> It’s not just up to the dev, but the libraries we use and whether or not we can easily
> upgrade (or remove) them to upgrade the php version.
By "upgrade" then, do you mean convert them into modules, or just be able to use them
as-is.
As I read it and am envisioning it, there would be no changes needed to be able to use them as-is.
> I think it would be a mistake to exclude old code and/or prevent templating. Not only are there
> now decades old code in some orgs, but how would you write an email sender that sent templated
> emails, provide html, generate code, etc? There has to be an output from the code to be useful.
Excluding old code or templates from modules would not exclude them from working as they currently
do outside modules. As I see it, modules would be more about exporting classes and functions, not
generating output per se.
So all that decades of old code could continue to exist outside modules, as it currently does today.
> I think it’s fine to use js as an inspiration, but it isn’t the only one out there. There
> is some precedent to consider directories as modules (go calls them “packages”) and especially
> in PHP where namespaces (due to PSR-4 autoloading) typically match directory structures.
Totally agree about inspiration for modules outside JS, but not sure that PHP namespaces are the
best place to look for inspiration.
Namespaces by their very nature were designed to enable autoloading with a one-to-one file to class
or interface, and by nature add conceptual scope and complexity to a project that would not be
required if a modern module/package system were added to PHP.
Modules could and IMO should be a rethink that learns the lessons other languages have learned over
the past decade+.
>> Node.js uses package.json and the attendant npm to do this sort of prep work. And
>> it's a critical part of this since modules can be versioned, and different modules may need to
>> run different specific versions of other modules.
>
> Please, please, please do not make a json file a configuration language. You can’t comment in
> them, you can’t handle “if php version <9, load this, or if this extension is installed, use
> this.”
>
> Maybe that is desirable, but doing things slightly different based on extensions loaded is def
> a thing.
I don't think commenting is important in this file, or even desired.
As I proposed above, these could be protobuf or phar. These should be build artifacts that can be
generated on the fly during development or for newbies even during deployment, not hand-managed.
I could see the generation of two files; one in binary form and one that is readonly so a developer
can double-check what is in the current protobuf or phar file.
>> Those are implementation details a little further down the road than we're ready for,
>> I think.
>
> Personally, if these are going to have any special syntax, we probably shouldn’t call them
> .php files. Maybe .phm?
I was going to suggest that, and then remembered earlier PHP when there were multiple file
extensions and that was a nightmare.
This does remind me to mention that I think there should be a required "module"
declaration at the top of each file just like Go requires a "package" declaration at the
top of each file. That would make it trivial for tooling to differentiate, even with grep.
> the only thing I don’t like about this import/export thing is that it reminds me of the days
> when we had to carefully order our require_once directives to make sure files were loaded before
> they were used. So, I think it is worth thinking about how loading will work and whether loading can
> be dynamic, hoisted out of function calls (like js), how order matters, whether packages can enrich
> other packages (like doctrine packages) and if so, how much they can gain access to internal state,
> etc. This is very much not “a solved problem.”
That is why I proposed having a "compiled" module symbol table to eliminate most (all?) of
those issues.
> On Jun 27, 2024, at 6:00 PM, Rowan Tommins [IMSoP] <[email protected]> wrote:
> I do think PHP badly needs a native concept of "module" or "package" - in
> fact, I'm increasingly convinced it's the inevitable path we'll end up on at some
> point. BUT I think any such concept needs to be built on top of what we have right now. That means:
>
> - It should build on or work in harmony with namespaces, not ignore or replace them
It may be an unpopular opinion, but I would argue that namespaces were optimized for autoloading and
the one class/interface per file paradigm, not to mention to regrettable choice of using the escape
operator to seperate namespaces and that fact that PHP throws away a lot of information about
namespaces at runtime.
IMO allowing modules to eventually deprecate namespaces — at least in a defacto form of
deprecation — would allow modules to be much better than if the try to cling to a less desirable
past.
> - It should be easy to take existing code, and convert it to a module/package
Maybe, but not if that means modules retain baggage that should really be jettisoned.
> and namespaces have proved an extremely successful way of sharing code without those names
> colliding.
At the expense of a lot more complexity than necessary, yes.
Managing symbols in a module need not be a hard problem if PHP recognizes modules internally rather
than trying to munge everything into a global namespace like with namespaces.
> Other parts of your e-mail are essentially an unrelated idea, to have some new
> "PHP++" dialect, where a bunch of "bad" things are removed. You're not the
> first person to be tempted by this, but I think the history HHVM and Hack is educational here:
> initially, PHP and Hack were designed to interoperate on one run-time, but the more they tried to
> optimise for Hack, the harder it became to support PHP, and now Hack is a completely independent
> language.
While I agree that some things are unnecessary — such as unifying scope resolution operators for
existing concepts — past failure does not guarantee future failure.
Hack tried to create an entirely new language yet still be PHP compatible. Learn from the Hack
experience and rather than create an entirely new language, PHP modules could simply add constraints
for code in modules, and then any "new" language features that are not module-specific
by-nature should be considered to work everywhere in PHP, or not at all.
> On Jun 27, 2024, at 6:41 PM, Larry Garfield <[email protected]> wrote:
> What problem would packages/modules/whatever be solving that isn't already adequately
> solved?
Not speaking for Michael, obviously, but speaking for what I envision:
1. Adding a module/package system to PHP with modern module features
- including module private, module function, and module properties
2. Providing an alternative to auto-loader-optimized namespaces.
- better code management and better page load performance
> Do we want:
>
> 1. Packages and namespaces are synonymous? (This is roughly how JVM languages work, I
> believe.)
> 2. Packages and files are synonymous? (This is how Python and Javascript work.)
> 3. All packages correspond to a namespace, but not all namespaces are a package?
I would argue packages (modules) should be orthogonal to namespaces to allow modules to be optimized
for what other languages have learned about packages/modules over the past decade+.
The fact that namespaces use the escape character as a separator, that PHP does not keep track of
namespace after parsing is enough reason to move on from them, and that they were optimize for
one-to-one symbol to file autoload are enough reasons IMO to envision a way to move on from them.
> And given the near-universality of PSR-4 file structure, what impact would each of those have
> in practice?
Orthogonal. Old way vs new way. But still completely usable, just not as modules without
conversion.
The fact PSR-4 exists is an artifact of autoloading single-symbol files and thus a sunken cost does
not mean that PHP should not cling to for modules just because they currently exist.
-Mike