On Thu, Apr 3, 2025, at 4:06 PM, Rowan Tommins [IMSoP] wrote:
> On 03/04/2025 18:06, Larry Garfield wrote:
>> So if we expect higher order functions to be common (and I would probably mainly use them
>> myself), then it would be wise to figure out some way to make them more efficient. Auto-first-arg
>> is one way.
>
> From this angle, auto-first-arg is a very limited compiler optimisation
> for partial application.
I'd say it has the dual benefit of optimization and ergonomics. (Though see discussion below.)
> With PFA and one-arg-callable pipes, you could add a parser rule that
> matches this, with the same output:
>
> $foo |> bar(?, $baz);
>
> But you'd also be able to do this:
>
> $baz |> bar($foo, ?);
>
> And maybe the compiler could optimise that case too.
From what Arnaud has told me, any PFA that has a single, fixed-position-number argument remaining
should be optimizable. (Though that's a task for whenever PFA is next worked on, if it is next
worked on.)
> Neither helps with the performance of higher order functions which are
> doing more than partial application, like map and filter themselves. I
> understand there's a high cost to context-switching between C and PHP;
> presumably if there was an easy solution for that someone would have
> done it already.
> On 03/04/2025 18:39, Ilija Tovilo wrote:
>> To me, pipes improve readability when they behave like methods, i.e.
>> they perform some operation on a subject. This resembles Swift's
>> protocol extensions or Rust's trait default implementations, except
>> using a different "method" call operator.
>> [...]
>> If we decide not to add an iterator API that works well with
>> first-arg, then I agree that this is not the right approach. But if we
>> do, then neither of your examples are problematic.
>
>
> I guess those two things go together quite well as a mental model:
> pipes as a way to implement extension methods, and new functions
> designed for use as extension methods.
>
> I think I'd be more welcoming of it if we actually implemented
> extension methods instead of pipes, and then the new iterator API was
> extension-method-only. It feels less like "one of the arguments is
> missing" if that argument is *always* expressed as the left-hand side
> of an arrow or some sort.
As I've noted, classic pipes (current RFC, unary function only) and extension functions are not
mutually exclusive, and I see no reason we couldn't add both. Auto-partialing first-arg pipes
and dedicated extension functions step on each other's toes a bit more, however.
To address both this and Ilija's email, I was toying with extension functions as a concept a
while back. I also did extensive research into "collections" in other languages last year
with Derick. (See discussion in a previous PHP Foundation report[1]). That led me to a number of
conclusions that I still hold to:
* A new iterable API is absolutely a good thing and we should do it.
* That said, we *need* to split Sequence, Set, and Dictionary into separate types. We are the only
language I reviewed that didn't have them as separate constructs with their own APIs.
* The use of the same construct (arrays and iterables) for all three types is a fundamental and core
flaw in PHP's design that we should not double-down on. It's ergonomically awful,
it's bad for performance, and it invites major security holes. (The "Drupageddon"
remote exploit was caused by using an array and assuming it was sequential when it was actually a
map.)
So while I want a new iterable API, the more I think on it, the more I think a bunch of map(iterable
$it, callable $fn) style functions would not be the right way to do it. That would be easy, but
also ineffective.
The behavior of even basic operations like map and filter are subtly different depending on which
type you're dealing with. Whether the input is lazy or not is the least of the concerns. The
bigger issue is when to pass keys to the $fn; probably always in Dict, probably never in Seq, and
certainly never in Set (as there are no meaningful keys). Similarly, when filtering a Dict, you
would want keys preserved. When filtering a Seq, you'd want the indexes re-zeroed. (Or to
seem like it, given or take implementation details.) And then, yes, there's the laziness
question.
So we'd effectively want three different versions of map(), filter(), etc. if we didn't
want to perpetuate and further entrench the design flaw and security hole that is "sequences
and hashes are the same thing if you squint." And... frankly I'd probably vote against an
interable/collections API that didn't address that issue.
However, a simple "first arg" pipe wouldn't allow for that. Or rather, we'd
need to implement seqMap(iterable $it, callable $fn), setMap(iterable $it, callable $fn), and
dictMap(iterable $it, callable $fn). And the same split for filter, and probably a few other
things. That seems ergonomically suspect, at best, and still wouldn't really address the issue
since you would have no way to ensure you're using the "right" version of each
function. Similarly, a dict version of implode() would likely need to take 2 separators, whereas the
other types would take only one.
So the more I think on it, the more I think the sort of iterable API that first-arg pipes would make
easy is... probably not the iterable API we want anyway. There may well be other cases for
Elixir-style first-arg pipes, but a new iterable API isn't one of them, at least not in this
form.
Which brings us then to extension functions. Pipes and higher order functions, or first-arg pipes,
can act as a sort of "junior" extension functions, but for the reasons listed above fall
short of being real extension functions.
For comparison, extension functions in Kotlin look like this:
fun SomeType.foo(a: Int) {
// a is a variable. "this" is the SomeType the function was called on.
// However, this is still "external" scope so only public members are usable.
}
val s = SomeType()
s->foo(5)
(Kotlin doesn't have a "new" keyword; the above is how you instantiate an object.)
Arguably, Go is entirely built as extension functions. It looks like this:
func (st SomeType) foo(a int) {
// st and a are both variables here. Do as you will.
}
Notably for us, the same function can be defined multiple times against different types. That
allows the system to differentiate between A.foo() and B.foo(). You can also attach extension
functions to interfaces. In fact, most of Kotlin's collections (list, set, map) API is
implemented as extension functions on interfaces, of which they have many.
However, both Go and Kotlin are compiled languages, which means the compiler has a complete view of
the code at compile time, and can sort out which extension function to use in a given situation
statically. That is, of course, not the case in PHP.
That means even if we figure out a way to define multiple foo() functions that apply to different
types, and can agree that doing so is not evil (some have argued it's too close to
function/method overloading, which they claim is evil; I disagree with both points), there is still
a very non-trivial task of figuring out how to resolve the function to call at runtime, probably
somehow leveraging autoloading, which also then runs us up against function autoloading, etc. I
hope that is a solvable problem, but I don't currently know how to solve it.
So "real" extension functions are an epic unto themselves, even though I really really
want them. (They are fantastically ergonomic for converting from one representation to another,
like from an ORM entity to a minimal struct to serialize as JSON, and vice versa. I quite miss them
from Kotlin).
It would be really nice if we could follow Kotlin's example and build 3 different collection
types (likely via objects), and then build most of the API for them in extension functions rather
than as methods. However, that sounds harder every time I dig into it.
As a side note to Yakov[2], a Uniform Function Call Syntax in PHP would have all the same problems
as extension functions, even before we get into the issue that Rowan, Tim, and others have brought
up that PHP is wildly inconsistent in having the "subject" first in a function call.
Without that UFCS doesn't make much sense. While I appreciate the elegance of it, in practice,
figuring out extension functions as a dedicated syntax (akin to Kotlin or Go above) is probably the
best we could do, if we can even do that.
All of which is to say... I think I may have talked myself back around to just using basic unary
function pipes and "suck it up" on the extra call for higher order functions for now,
unless someone can show a fair number of non-iterable use cases where it would be helpful. That
then would unblock the other incremental improvements listed in the RFC (compose, PFA, and
$$->foo()). True extension functions could then be explored later (likely by people with way
more engine knowledge than me) as their own thing, whether using ->, +>, or something else
entirely. We just need to agree that the existence of pipes does not render extension functions
moot.
Thoughts?
--Larry Garfield
[1] https://thephp.foundation/blog/2024/08/19/state-of-generics-and-collections/
[2] https://externals.io/message/127037