> On Jun 28, 2024, at 10:12 AM, Rowan Tommins [IMSoP] <[email protected] <mailto:[email protected]>> wrote:
> Namespaces don't require autoloading, and autoloading doesn't require one file per
> class.
No they do not, but the design of each was heavily intertwined with each other resulting in a less
than optimal design, IMO.
So, are you arguing to keep one and eject the other for modules, and if so which are you arguing we
eject? Autoloading?
Or are you arguing to keep both for modules, in which case your argument above is moot?
> To compile a program with multiple source files, in any language, you need one of two things:
>
> a) A list of files you want to compile. Maybe auto-generated, maybe done with a recursive
> iteration over a directory, but ultimately the compiler needs a file path to process.
Recursion is only needed if modules are hierarchical in nature.
> b) A way for the compiler to tell, based on some symbol it wants to resolve, which file should
> be compiled.
That presumes the compiler did not simply generate an AST from the list of files.
> PHP originally provided only option (a), via the include and require keywords. Autoloading adds
> option (b), where you provide a function which takes a class name and does *whatever you want* to
> find the definition.
And that "whatever you want" takes execution time (and tracing through when you are
debugging.) But when you look at many other languages loading is an implementation detail that PHP
chose to hoist onto userland developers when PHP could have established the rules to handle it more
performantly without userland involvement.
Or is there some aspect of autoloading that could not be handled by PHP itself? Note I am asking
only within the propose scope of modules, which we could constrain to optimize their runtime use.
> I think it might be time to re-visit the tooling around option (a), as OpCache makes the cost
> of eagerly loading a list of files much lower than it was when autoloading was added.
Now you are getting somewhere.
Imagine that each module — which could equal a single directory — could have a pre-compiled
op-cache which is essentially what I proposed in other recent emails.
> That could be as simple as include_all($directory), or as fancy as
> include_from_manifest_file($some_crazy_binary_file_format); either could be implemented right now in
> userland, because it all eventually comes down to calling include or require.
meh.
That sounds like a way to avoid discussing the ways in which smartly designed modules could really
improve PHP.
>>> My opinions match Larry's almost exactly: I want package-level optimisation, and
>>> package-private declarations. But I don't want to rewrite my entire codebase to start using a
>>> completely different naming system.
>> I can't see how package-privates would be of any value to you *unless* you rewrite
>> your codebase.
> Simple: I have private Composer packages, right now, that group all their classes under a
> particular namespace prefix. I want to be able to mark some classes in that namespace as
> "internal".
Not simple, although I admit I am being pedantic about words used here, but for a reason.
I asked about "package-privates," you responded with "namespace-privates."
Adding private to namespaces is orthogonal to the discussion of packages.
To require that packages be constrained to have all the same warts as namespaces and existing PHP
code simply so you can have namespace-privates is short-sighted (and IMO a bit selfish.)
Alternately, namespaces could get private scope in parallel to having modules be considered.
That would allow modules to gain improvements that we could not get by having to maintain BC with
namespaces.
Which causes me to ask: If you have really wanted namespace private why has it been six years since
it was even last mentioned on the list, and four years since last discussed?
https://externals.io/message/101323 <https://externals.io/message/101323>
Why has there not been an RFC since this one https://wiki.php.net/rfc/namespace-visibility
<https://wiki.php.net/rfc/namespace-visibility>
six years ago, that was not even voted on?
Why is it that when the topic of addressing modules/packages comes up — which has been talked
about numerous times in the past six years — do you now bring up namespace privates in a manner
that would effectively torpedo goals of the modules discussion, at least from the perspective of the
OP and myself?
If namespace private were really something important to you, why haven't you championed it
before, rather than hijack a discussion about the benefits we could get from modules if not
constrained by namespaces?
> I do not want to change every place that uses the existing classes to reference
> "ModuleName@ClassName" instead of "NamespacePrefix\ClassName", or change every
> "use" to "import from".
Then don't. Champion this RFC https://wiki.php.net/rfc/namespace-visibility
<https://wiki.php.net/rfc/namespace-visibility>
and get what you want.
But please don't argue against a discussion on modules because you want a feature that can be
gotten orthogonally. (If you must argue against it, make arguments for which accommodations for your
preferences cannot be found.)
> Code doesn't existing in isolation; if Symfony Mailer is re-published as a
> "module", every single application that uses it needs to change their code from
> referencing namespaced classes, to having "import" statements.
And that is bad, how?
But before you answer, it just means that instead of a use statement in your existing
code you change to a import statement.
You'd then of course need to changes — if applicable — to call the new Symphony Mailer, but
you'd have to do that with or without modules.
Or is there something else I am missing?
>> As for package-level optimisation, you'll need to give examples of what you mean there
>> as I don't want to wrongly assume.
>
> Currently, OpCache only optimises per file, because it can't guarantee how files will be
> used together.
>
> A simple example is function fallback: if you could declare a package as "completely
> loaded", OpCache could replace all references to "strlen" with "\strlen",
> knowing that no namespaced function with that name could be added later.
Thank you for elaborating on that.
So, champion an RFC to improve OpCache for namespaces. That need not impose on the discussion about
modules.
Further, and this is what is nice about being able to discuss modules not having to be compatible
with namespaces, if there are aspect of namespaces that make optimization hard or impossible then we
could potentially set up rules of modules that make similar optimizations easy and/or possible.
EVEN further, consider the fact that in PHP all class members are public by default. One thing we
could have in modules is to go back to short var and eliminate both
private and protected modifiers and only have public with the
default behavior being what is private outside of modules. protected would
no longer be needed as we would have module scope which is defacto-protected.
Classes could be final by default in modules and then we could modify them with an
open keyword (thanks to Lynn for that one.)
And so on. In other words, if we could treat modules as their own sandbox, we could get fix many of
the regrettable former design choices of the PHP language — some of which are to make PHP be
beginner friendly — and potentially re-energize people who once looked at PHP and dismissed it to
give it another look.
> What I meant was: we can't just treat namespaces and modules as completely separate
> things, and assume that every code file will be using one style or the other. We have to imagine the
> user experience when there is a mix of the two.
Why can we not just treat namespaces and modules as completely separate things?
> I can't imagine it being pleasant to have a mix of "import" and "use"
> statements at the top of a file, with different and even conflicting semantics.
That feels like a frivolous concern when compared to the benefits we could see with modules,
especially when there would be ways to mitigate your stated concerns here.
If you don't like to see imports, but your imports in a namespace and then "use" that
namespace.
Or we could allow "use module" instead of (or in addition to) "import" and then
it could look more pleasant for you.
As for conflicting semantics:
1.) I'm not seeing how those could be significant in the using/importing file, and
2.) Isn't dealing with conflicting semantics just a part of programming?
3.) Don't "use" and "use function" have conflicting semantics?
God knows that "use" by itself has many confusing semantics, which "import"
could avoid.
> Perhaps I didn't word the question well. What I'm really asking, as someone
> who's never used JS or Go modules, is why I'd want to write "import", rather
> than referencing a global name in a hierarchy.
"use module" would work just as well as "import"; the "import" is not
special, the module scoping and features are what is valuable here.
For specifics see my other recent emails on the subject. If they do not explain, please ask again
with specifics.
> That's really all I mean by "making it compatible with namespace": I want
> "new \Foo\Bar\Baz;" to be able to refer to a "packaged" class, probably by
> having a way to mark all classes under "\Foo\Bar" as belonging to a particular package.
And that is what I am trying to get away from.
First the backslash — because when using in reflection or other dynamic programming they have to
be escaped which can lead to escaping errors. I know you don't care, but I and others do.
Second, the hierarchy. Because there is no constraint on hierarchy PHP subtly encourages developers
— as if sirens of the Odyssey — to create large hierarchies. I even find myself doing it as I
fighting myself against it.
The reasons hierarchy is bad is:
1.) larger hierarchies grow conceptual complexity,
2.) they place no limit on package growth as you can always create subdirectories,
3.) they make it harder to "see" all the code files in one place (a single directory),
4.) they constrain where code is located when there are benefits to a different layout
> That's really all I mean by "making it compatible with namespace": I want
> "new \Foo\Bar\Baz;" to be able to refer to a "packaged" class, probably by
> having a way to mark all classes under "\Foo\Bar" as belonging to a particular package.
Revisiting this, why is it important to you that "new \Foo\Bar\Baz" refer to a
"packaged" class vs a namespaced class, assuming you had namespace-private and OpCache
improvements?
Why can't you still just use the namespaces you prefer and let "packages" (modules)
improve in other ways?
I am trying my best not to make this ad-hominem so forgive me but I do have to ask if this is just
not a case of "I am comfortable doing it the way I have been doing it and do not want to
consider changing," maybe? Note I am asking that question limited to the one statement I
quoted above, not on the broader discussion.
>> 6. Modules and packages are typically small-scoped to a single directory and it is a code
>> smell to have many different packages tightly coupled, as is the case with namespaces. Forcing
>> modules to munge with namespaces would mean most modules would be written with those code smells for
>> years because that will be how everyone doing a quick shift from namespace to module will write
>> their modules.
>
> Again, this is entirely about code style, and not something the language can control.
A language cannot control it, but a language can encourage or discourse it.
And the PHP language encourages a large amount of file and directory bloat.
One only need to compare the number of files in most PHP libraries to the number of files in JS or
Go package to see that the nature of a language clearly does not influence.
To bring stats vs. opinion I asked ChatGPT what the two equivalent packages are to Symphony for JS
and Go respectively and it suggested ExpressJS and Gin. So I cloned them to see the number of files
and directories each has. From the root of each repo:
Project Files Dirs
Symfony: 12,504 2,162
ExpressJS: 259 87
Gin(GoLang): 145 30
The comparison might not be completely fair given how much longer Symfony has been around, but they
all target the same use-case so even if there is less functionality in ExpressJS or Gin.
Given that I think that well over an order of magnitude more files is a really odiferous code smell,
and is thanks to the language which admittedly cannot "control" layout, but definitely
influences it.
Am I wrong? Present any other relatively equivalent project comparisons you please. Here are the
bash commands to count files and dirs:
find /path/to/subdirectory -type f | wc -l
find /path/to/subdirectory -type d | wc -l
> Also, the JS insistence on having a separate package for every tiny function is a common source
> of criticism, so personally I am very happy that PHP packages are generally larger than that.
I can't speak for the OP, but nothing I am proposing is advocating for separate packages for
every tiny functions. Nothing.
Instead I am advocating for packages that are mostly in a few directories instead of almost two
magnitudes more!
>> That said, maybe the best solution is to NOT put the stake in the ground right now and say
>> "They must be namespace compatible" or "They must not be namespace compatible"
>> but move forward with an open mind so that we can tease out exactly how namespaces would constrain
>> modules and and then make the decision later for what would be in the best interest of moving PHP
>> into the future.
>
> If and when an actual problem arises, let's discuss it.
Not "problems" but instead "opportunities."
I have already pointed out numerous opportunities in this email and one of my recent emails.
>>> What specifically stops us doing all the things you've been discussing around
>>> loading, and visibility, etc, in a way that's compatible with the
>>> 400_000 packages available on Packagist, and billions of lines of existing code?
>>
>> You speak as if I am proposing getting rid of namespaces and making those 400_000 packages
>> available on Packagist, and billions of lines of existing code not work in PHP. Of course not.
>
> No, I'm saying that every one of those packages could benefit if we make incremental
> changes.
Maybe.
What benefits can you envision you would get if PHP made namespaces==modules compared with the
benefits I have mentioned for making modules not be constrained to compatibility with namespaces
(besides private and OpCache as we already discussed you pursue for namespaces?)
Can we get precompiling for modules in a directory and written to a .php.module file?
We can't do that with namespaces because scanning recursively could take too long at runtime.
Can we get default private for all symbols and class members in namespaces? No, that would be a huge
BC break.
Can we get namespaces to be first-class AST participants? If yes, why have we not done it before?
I could go on, but this email is getting loooong.
> I don't want to couple it so that you can't have "package private" without
> also switching to some new "advanced" dialect of the language, and I don't see any
> reason why we need to do so.
And I am not advocating that. I am advocating you should get "namespace private." Hey RFC
is already written! https://wiki.php.net/rfc/namespace-visibility
<https://wiki.php.net/rfc/namespace-visibility>
And most of the other benefits of modules as I am proposing would be BC breaks so you could not get
them in namespaces anyway.
Unless you can come up with something besides private and opCache I had not considered.
> Maybe package scoped declares could allow opting in to certain checks, but I don't think
> "is in a package" and "has been audited for a load of extra breaking changes"
> should be set by the same flag.
I am not aware of any discussion of opting in, flags, nor auditing with respect to modules.
-Mike