A significant source of motivation for the pilot wave framework pursued here is derived from a mechanistic, “how could this possibly work” perspective on the quantum world. This is not a naive question at the level of an aether or some other kind of “mechanical” substrate for quantum phenomena; instead, the quest for a universal basis representation that can solve any quantum problem provides an information-theoretic framing of what Nature must somehow be accomplishing. In this page, we consider more specifically how the physical structure and properties of things like EM waves, quantum wave functions, and discrete, localized particles might all interact with each other, at the level of how the relevant information is distributed and would be available to drive the known physical processes.
The point of departure is the pure-wave based picture at the foundation of the Standard Model description, which can be seen as providing a clear answer to certain critical such mechanistic questions, while leaving other aspects entirely unexplained. Hopefully this grounding in such a solid foundation provides some appreciation for the importance of the remaining gaps at this level of description.
Specifically, consider the gauge theory description of the minimal coupling of the electromagnetic (EM) field (i.e., Maxwell’s equations) with a wave of charge described by the Dirac wave equation. This provides a complete account for how long-range EM forces can interact with a charged particle like an electron, where the spatially-distributed nature of the Dirac wave can plausibly interact with the spatially-distributed EM wave to exert force, bidirectionally (i.e., the charge current from the Dirac wave provides the source of the EM waves, and the EM waves in turn exert force on the Dirac waves).
By avoiding discrete, point-like particles, the distributed, conserved charge current sourcing the EM field provides a solution to the otherwise challenging and largely unresolved back-reaction problem. Furthermore, the ability of the gauge theory framework to ensure that this bidirectional interaction exhibits critical conservation properties is a major advantage of this wave-wave interaction mechanism.
Thus, at a qualitative level, this purely wave-based interaction seems to provide a suitable universal basis representation: the physical properties of these waves are mutually compatible, and there is no obvious impediment for this system of interacting distributed waves to properly capture the basic physical phenomena of mutual force interactions.
However, this same property that makes waves so natural for communicating interacting forces also makes them bad at representing any kind of localized, unitary, conserved properties, because the wave mass continuously diffuses into an increasingly thin and widely distributed film. This is specifically a problem for the quantum wave associated with a particle like an electron, which is inevitably found to have a point-like discrete existence whenever it is measured.
Furthermore, electrons can collide with other particles at high energies (in nature and in high-energy colliders), and end up transforming into other particles. The rules for these transformations involve strict forms of conservation, where discrete properties like charge, spin, and lepton-vs-quark status are conserved in various ways. The possibility that a widely-distributed Dirac wave could somehow be gathered back up to then be localized and appropriately converted into the relevant other particles represents a major information-level problem for this wave-only universal basis representation.
The conceptual problem with recovering a discrete, localized particle from a distributed quantum wave is indeed identical to the paradoxical and unexplained process invoked in the notion of measurement in the standard Copenhagen interpretation.
The solution to this problem provided by the pilot-wave approach is to introduce an always-localized, discrete particle that is somehow coupled to its corresponding quantum wave, and whose motion is guided by this wave, which is in turn influenced by the EM force fields. Because all measurements can be ultimately reduced to localization measurements of massive particles, the presence of this discrete particle can in-principle provide a reasonable explanation for why wave collapse is not needed for measurement, and for the high-energy particle transmutation phenomena.
Thus, given all the advantages of the wave-wave model for particle and force-field interactions, it makes sense for the quantum wave to interact with the force field, and then the quantum wave in turn interacts with the discrete particle. But this just pushes the problem off one step, because the same issues that arise with a discrete particle interacting with a force-field wave (i.e., EM), now migrate to the particle interaction with its own quantum wave.
Thus, there are fundamental issues associated with any scenario involving a discrete, point-like particle interacting with any kind of wave.
How does the discrete particle ensure that its corresponding quantum wave does not dissipate into oblivion, and that it remains properly coupled to the particle? For example, during a particle transmutation event, the resulting quantum wave would need to be updated appropriately for the new type of particles that have emerged. This question of the particle-to-wave back-reaction was raised by de Broglie in his double solution version of the pilot wave framework, but nobody has yet made a fully workable version of such a model that is consistent with standard quantum physics (see pilot-wave for extensive discussion).
In particular, if the particle serves as a source for the quantum wave, then the problem of very high field strengths in the immediate neighborhood of a discrete particle arises. Furthermore, in the cellular automaton framework, the discrete particle needs to make discrete jumps to a neighboring cell when it moves (via stochastic motion), and such jumps would create very high-amplitude, high-frequency disturbances in the force field generation.
As with the hydrodynamic analog of the pilot wave system, the particle may instead provide a kind of driving oscillation to the quantum wave field, and this wave field may naturally decay over some Compton-length level distance. Thus, the sourcing of the quantum wave may take a more subtle form that ends up avoiding the issues that arise in the EM force-field situation.
In any case, aside from these remaining issues, the natural conclusion from the above mechanistic considerations is that waves should be used for forces (i.e., bosons) and discrete particles coupled with quantum waves (i.e., the pilot-wave model) for massive particles (fermions). The retention of classical force fields is the essential feature of the semiclassical framework.
The semiclassical approach can be more strongly motivated by thinking in greater detail about how an alternative, particle-based ontology would work in the context of force interactions. If you just have a simple discrete point-like particle sitting in some kind of lattice-like grid, it is very difficult to implement realistic force-field interactions among such particles, especially when using other discrete particles like photons to mediate these interactions. Do we really imagine that an electron is constantly spewing baseball-like photons out in all directions to hit other electrons? How does such a scheme ever achieve any kind of smooth field-like coverage of space using discrete point-like entities? How many balls per femtosecond does it have to spew? How do they manage to spread out uniformly over space and time, while properly conveying the dynamic interactions among the magnetic and electric aspects of the wave functions?
As discussed in the photon page, the way in which a photon is actually represented in standard quantum field theory is purely in terms of its energy and frequency properties, with absolutely no localization properties at all. Thus, the photon of this framework is quite unlike any kind of discrete particle that we think of for the particle aspect of the pilot-wave framework. Indeed, the same holds for the way that fermion particles are represented in the standard field theory framework as well. But whereas it does make physical sense to introduce discrete localized particles for massive fermion particles in a pilot-wave framework, it really doesn’t make sense to also do this for force fields.
The implication of this is that the photon as a particle-like concept does not exist, even though a localized wave-packet type of EM wave is perfectly reasonable.
See challenges for a discussion of the remaining challenges for achieving a universal basis representation that provides a sensible mechanistic basis for quantum physics.