To this day, manipulation still stands as one of the hardest challenges in robotics. In this work, we examine the board game Dr. Eureka as a benchmark to encourage further development in the field. The game consists of a race to solve a manipulation puzzle: reordering colored balls in transparent tubes, in which the solution requires planning, dexterity and agility. In this work, we present a robot (Tactical Hazardous Operations Robot 3) that can solve this problem, nicely integrating several classical and state-of-the-art techniques. We represent the puzzle states as graph and solve it as a shortest path problem, in addition to applying computer vision combined with precise motions to perform the manipulation. In this paper, we also present a customized implementation of YOLO (called YOLO-Dr. Eureka) and we implement an original neural network (NN)-based incremental solution to the inverse kinematics problem. We show that this NN outperforms the inverse of the Jacobian method for large step sizes. Albeit requiring more computation per control cycle, the larger steps allow for much larger movements per cycle. To evaluate the experiment, we perform trials against a human using the same set of initial conditions.