A Grand Unified Learning Algorithm?


In what sense can we articulate a single learning algorithm or meta-algorithm, encompassing all the types of learning that an AGI system needs to do?

I opined on this topic in a paper I gave at the AGI-17 conference in Melbourne, which won the Kurzweil Best Idea Prize there,


The thoughts there build a lot on these ideas


which are likely to be implemented (in some, maybe much improved form) during the next couple years as OpenCog AI gets more sophisticated

In short I think there is a universal learning meta-algorithm, that I call “Probabilistic Growth and Mining of Combinations” (PGMC), and I think the ways PGMC manifests itself in different domain-specific ways, can be mapped into each other morphically in elegant ways… Leveraging these insights in the context of practical AGI system design is another issue…


in addition to cognitive synergy (which btw is an exciting expression) our general intelligence (I think) comes from our ability to learn how to learn. As soon as we see something new, we can automatically mine its unique features and memorise and get better with experience. If a neural network could be made to learn how to build efficient and application specific neural networks, it could fall into a vortex of discoveries learning everything new.


PGMC seems like it uses similar ideas to my Heuristic Algorithmic Memory which was the first to integrate frequent subprogram mining into a meta-learning algorithm. (AGI 2010,2011,2014). Although it’s just a sketch. Who knows, maybe there are several universal meta-learning algorithms that are equally well or complementary. We’ll see I guess but I don’t disclose algorithms before publication :slight_smile:


Google AI already implemented very cool versions of Schmidhuber’s ideas on meta-learning for ANN’s. I wonder if you’ve seen the latest papers. What I use meta-learning is for MEMORY but there might be other uses. In general, I hypothesize that meta-learning is the crucible of CONSCIOUSNESS. Yes, the big C. In latest Deep Mind experiments, you can see how the prefrontal cortex model as a meta-learning system worked surprisingly well. Knowing something about your brain is Minsky’s model of C. Though the most sophisticated such theory was advanced by Friston. Follow my Twitter http://twitter.com/examachine , I do post new papers about this subject. BTW, we can definitely implement the newest algos that use RL for meta-learning and it looks like Schmidhuber does approve of that approach.


I think AI is the first invention, or realization, that most likely depends on unified theory to the extent that, by creating a finished product, even if imperfect, will be a catalyst to the establishment of a unified template for all future processes.
We have in place, established compartments such as the periodic table, where chemical compounds could be viewed as the cognitive synergy of that model.
Likewise, the study of time displays a similar model, where specific function blends from one to another, yet also relies on indirect interaction with separate functions located throughout the model, to establish ideas and shapes. The flow of time accomplishes this by layers of finite recurring sets, moving in separate directions. Sort of like the spinning wheel or propeller that appears stationary.
For example, if I plug Jung and Freud into this system for analysis, I find that in the 260 section consciousness model, moving counterclockwise, they are born on the exact same day. 1 out of 260. But, in the other model of 36 sections, moving clockwise, they are in different compartments. By function, their similarities are illustrated and known in the 260, and the difference in the 36 illustrates their differences. If we had known that, Freud’s sexual content could have been compartmentalized into the proper perspective, and years of needless discussion and angst would have been avoided. His projection differentiated from the needed information.
If areas of learning, are paired with inherent talents of a given set, then it must also correlate with function, and periodicity. These in turn must also conform to known processes that differentiate between material processes and consciousness processes. Calendars.
We are arriving at a product driven by a process. I would earnestly explore that process, to duplicate it. If there is a process and model that predicts the creation of AI and robots, in space and time, then it would be capable of predictive abilities beyond our imagination. The idea is generated in one wheel, and the process dictated in another, such that when they align, the product becomes manifest.


Reinventing the wheel


Rediscovery of existing wheels and finding practical applications.


That is what field trials are indicating. Yes.


Fascinating insights. Thanks for sharing. Later :v:t5: