Reinforcement Learning Based Scheduling for the Next-Generation Astronomy Observation

The next-generation Very Large Array (ngVLA), as the next-generation radio/mm telescope operated by the National Radio Astronomy Observatory (NRAO), has ten times the sensitivity and spatial resolution of the previous generations of telescopes and can operate at frequencies spanning up to 116 GHz, which can provide large scale astrophysical imaging for new discovery. There are five ngVLA key science goals: i) unveiling the formation of solar system analogues, ii) probing the initial conditions for planetary systems and life with astrochemistry, iii) charting the assembly, structure, and evolution of galaxies from the first billion years to the present, iv) using pulsar in the galactic center as fundamental tests of gravity, v) understanding the formation and evolution of stellar and supermassive black holes in the era of multi-messenger astronomy.

To achieve the scientific goals, the next step is to extend the scheduling algorithm from the previous generation telescope to perform optimal or near-optimal scheduling solutions for ngVLA. Given antennas, subarrays were created that can be sufficient to cover all high-priority observation requests, then given the selected Scheduling Blocks (SBs) as observation task instances with multiple attributes including frequency, local standard time (LST), Phase Root Mean Square (RMS), weather-related attributes, angular resolution, largest angular scale, etc, the scheduling system arranges them with optimal or near-optimal performance to execute in the given observation time window. The system may first find eligible SBs and corresponding SAs according to the current and forecast weather conditions, then use a scheduling algorithm to generate subarrays eligible for selected SBs and a schedule that considers priorities. Previous scheduling approaches are not sufficiently efficient for ngVLA. Take the brute force method as an example. Searching all possible scheduling combinations is guaranteed to find the optimal scheduling plan, however, the scheduling overhead increases exponentially when the number of SBs increases and the searching is even more complex with the searching of all possible subarray combinations. To have an efficient solution in the long run, we compare multiple scheduling algorithms from the literature and propose to explore the use of abstract structure with policies to generate optimal or near-optimal algorithms, and leverage the off-line Reinforcement Learning (RL) method to solve the scheduling problem. In this work, we propose using Action Space, which generates all possible scheduling options given the states rather than comparing each SB. We also provided two conventional methods that have optimal or near-optimal performance with three metrics: throughput, cost, and job latency. Lastly, we leverage the RL method to achieve near-optimal or optimal performance compared to the conventional method. When scaling the number of antenna subsets, we find the prediction time of the RL method stays nearly constant while the conventional method has a prediction time that grows exponentially, we found the RL method can greatly reduce execution time compared to conventional methods starting 8 or more antenna subsets while maintaining the highest average performance.