Q-learning is a relatively new learning theory that is often said to be model free, because it doesn’t contain a model of its environment in the typical fashion. In the MPCR Lab, we have one of the world’s first implementations of a q-learning agent in the real world. OpenCv was used (with Python) to drastically reduce the state-space from N-dimensional to only 3 dimensions. Soon, we will be attempting to implement q-learning in drones.
Artificial Neural Networks
At the Center for Complex Systems and Brain Sciences, we are interested in understanding how the human brain works with the nervous system to perceive stimuli and produce an appropriate behavior in order to reproduce or provide similar mechanisms in computers. Artificial neural networks are biologically inspired systems that have long been used in machine learning. This method is very different from some of the traditional approaches to machine learning, in which the inputs and behaviors were hard-coded into the machine in a top-down fashion and expert systems were used. Artificial neural networks work in the opposite way and require much less instruction. For example, in the past, researches attempted to get computers to recognize different numbers by describing what the numbers looked like. This method didn’t work for two main reasons: it’s very hard to describe what a number looks like (if you don’t believe me, try and describe the number 2 so a computer can recognize it) and it takes too many rules to do this. Artificial neural networks have made great progress in this area, and allow for less instruction by picking up on patterns in the image. Another advantage is that they allow for learning and adaptation as the machine explores its environment, so it’s a dynamic system. Thus, these networks truly allow us to replicate human development and behavior through supervised or unsupervised learning. We are currently researching biologically plausible methods of perceiving and recognizing natural images using neural networks in conjunction with neurally plausible algorithms such as LCA (Locally Competitive Algorithms).
Sparse Coding/ Dictionary Learning
In 2007, the amount of data created began to surpass the Earth’s hardware storage capacity, and it has since increased in a near exponential fashion. It is projected that, in 2018, nearly a million minutes of video content will cross the world’s network every second. In sparse coding, an image or signal is broken down into patches, each containing 64 pixels typically. Dictionary learning is a powerful tool that uses l1 optimization or LCA to create what’s called a sparse dictionary of these patches that can be used in different amounts to reconstruct any image it was trained on. These dictionary atoms can then be used to help identify certain objects using l1 pooling, which we are doing in the MPCR Lab.
Lake Okeechobee is one of the most biodiverse systems in the world. It serves as a haven for a multitude of aquatic and terrestrial species, including very many species of wading birds. The health of this system and its inhabitants is at risk due to the invasion of exotic species. The economic cost of such invasions globally is billions of dollars every year. The aquatic applesnail, Pomacea maculata, is an example of an invasive species with established populations in Lake Okeechobee. It has also been introduced to Southeast Asia and decimated rice and taro fields, as a result of its huge appetite for aquatic vegetation. In Lake Okeechobee and South Florida, its impact seems to be less severe (with the exception of a few small, managed wetlands) at this point. Although this snail’s population in South Florida continues to see substantial growth in South Florida aquatic systems, its native relative, the Florida Applesnail, can be found in sparse populations throughout the region. This is a cause of concern, as the latter is the main food source of the federally endangered Everglades Snail Kite, and it is unclear at this time whether the invasive applesnail is an equally suitable food source for the bird. Much of my research in Lake Okeechobee has been to study and elucidate the factors and conditions that have allowed Pomacea maculata to propagate throughout Lake Okeechobee and the Florida Everglades in an attempt to provide some sort of possible control method.