• Rural Digital Europe
• Open Access close
• Publications close

The current success of Reinforcement Learning algorithms for its performance in complex environments has inspired many recent theoretical approaches to cognitive science. Artistic environments are studied within the cognitive science community as rich, natural, multi-sensory, multi-cultural environments. In this work, we propose the introduction of Reinforcement Learning for improving the control of artistic robot applications. Deep Q-learning Neural Networks (DQN) is one of the most successful algorithms for the implementation of Reinforcement Learning in robotics. DQN methods generate complex control policies for the execution of complex robot applications in a wide set of environments. Current art painting robot applications use simple control laws that limits the adaptability of the frameworks to a set of simple environments. In this work, the introduction of DQN within an art painting robot application is proposed. The goal is to study how the introduction of a complex control policy impacts the performance of a basic art painting robot application. The main expected contribution of this work is to serve as a first baseline for future works introducing DQN methods for complex art painting robot frameworks. Experiments consist of real world executions of human drawn sketches using the DQN generated policy and TEO, the humanoid robot. Results are compared in terms of similarity and obtained reward with respect to the reference inputs.

Somatosensation greatly increases the ability to control our natural body. This suggests that supplementing vision with haptic sensory feedback would also be helpful when a user aims at controlling a robotic arm proficiently. However, whether the position of the robot and its continuous update should be coded in a extrinsic or intrinsic reference frame is not known. Here we compared two different supplementary feedback contents concerning the status of a robotic limb in 2-DoFs configuration: one encoding the Cartesian coordinates of the end-effector of the robotic arm (i.e., Task-space feedback) and another and encoding the robot joints angles (i.e., Joint-space feedback). Feedback was delivered to blindfolded participants through vibrotactile stimulation applied on participants’ leg. After a 1.5-hour training with both feedbacks, participants were significantly more accurate with Task compared to Joint-space feedback, as shown by lower position and aiming errors, albeit not faster (i.e., similar onset delay). However, learning index during training was significantly higher in Joint space feedback compared to Task-space feedback. These results suggest that Task-space feedback is probably more intuitive and more suited for activities which require short training sessions, while Joint space feedback showed potential for long-term improvement. We speculate that the latter, despite performing worse in the present work, might be ultimately more suited for applications requiring long training, such as the control of supernumerary robotic limbs for surgical robotics, heavy industrial manufacturing, or more generally, in the context of human movement augmentation.

NBS provides the technical basis for adaptation to climate change, and co-creation is the vehicle for the co-production of knowledge and innovation, both forming a strong binomial for the UE Green Infrastructures Strategy. Nonetheless, one of the main challenges for the implementation of effective co-creation strategies is the incorporation of knowledge from diverse social systems. Knowledge production has been approached through different methodological models, such as the quintuple helix innovation by Carayannis, or the diffuse/expert knowledge model by Manzini. These theoretical models are based on linear knowledge transfers, without sufficiently depicting alternative knowledge flows among (un)conventional actors. In view of these limitations, the research proposes a third strategy: the KREBS cycle of creativity defined by Oxman is a conceptual map capable of describing knowledge transfers across the four modalities of human creativity (i.e., science, engineering, design, and art). Providing sufficient “creative energy” in a co-creation process would guarantee the successful production of knowledge. Thus, the research seeks to illuminate different co-creation strategies to promote “creative energy” in the design of the Cyborg Garden (CG) in Madrid, giving a novel application to Oxman’s methodological framework based on the Carayannis’s and Manzini’s models.

Allocentric semantic 3D maps are highly useful for a variety of human–machine interaction related tasks since egocentric viewpoints can be derived by the machine for the human partner. Class labels and map interpretations, however, may differ or could be missing for the participants due to the different perspectives. Particularly, when considering the viewpoint of a small robot, which significantly differs from the viewpoint of a human. In order to overcome this issue, and to establish common ground, we extend an existing real-time 3D semantic reconstruction pipeline with semantic matching across human and robot viewpoints. We use deep recognition networks, which usually perform well from higher (i.e., human) viewpoints but are inferior from lower viewpoints, such as that of a small robot. We propose several approaches for acquiring semantic labels for images taken from unusual perspectives. We start with a partial 3D semantic reconstruction from the human perspective that we transfer and adapt to the small robot’s perspective using superpixel segmentation and the geometry of the surroundings. The quality of the reconstruction is evaluated in the Habitat simulator and a real environment using a robot car with an RGBD camera. We show that the proposed approach provides high-quality semantic segmentation from the robot’s perspective, with accuracy comparable to the original one. In addition, we exploit the gained information and improve the recognition performance of the deep network for the lower viewpoints and show that the small robot alone is capable of generating high-quality semantic maps for the human partner. The computations are close to real-time, so the approach enables interactive applications.

Abstract We study whether or not a group of biomimetic waggle dancing robots is able to significantly influence the swarm-intelligent decision making of a honeybee colony, e.g. to avoid foraging at dangerous food patches using a mathematical model. Our model was successfully validated against data from two empirical experiments: one examined the selection of foraging targets and the other cross inhibition between foraging targets. We found that such biomimetic robots have a significant effect on a honeybee colony’s foraging decision. This effect correlates with the number of applied robots up to several dozens of robots and then saturates quickly with higher robot numbers. These robots can reallocate the bees’ pollination service in a directed way towards desired locations or boost it at specific locations, without having a significant negative effect on the colony’s nectar economy. Additionally, we found that such robots may be able to lower the influx of toxic substances from potentially harmful foraging sites by guiding the bees to alternative places. These effects also depend on the saturation level of the colony’s nectar stores. The more nectar is already stored in the colony, the easier the bees are guided by the robots to alternative foraging targets. Our study shows that biomimetic and socially immersive biomimetic robots are a relevant future research target in order to support (a) the bees by guiding them to safe (pesticide free) places, (b) the ecosystem via boosted and directed pollination services and (c) human society by supporting agricultural crop pollination, thus increasing our food security this way.

Abstract: Robotic technologies have shown the capability to interact with living organisms and even to form integrated mixed societies comprised of living and artificial agents. Bio-compatible robots, incorporating sensing and actuation capable of generating and responding to relevant stimuli, can be a tool to study collective behaviors previously unattainable with traditional techniques. To investigate collective behaviors of the western honeybee (Apis mellifera), we designed a robotic system capable of observing and modulating the bee cluster using an array of thermal sensors and actuators. We initially integrated the system into a beehive populated with approximately 4,000 bees for several months. The robotic system was able to observe the colony by continuously collecting spatio- temporal thermal profiles of the winter cluster. Furthermore, we found that our robotic device reliably modulated the superorganism’s response to dynamic thermal stimulation, influencing its spatiotemporal re-organization. In addition, after identifying the thermal collapse of a colony, we used the robotic system in a “life-support” mode via its thermal actuators. Ultimately, we demonstrated a robotic device capable of autonomous closed-loop interaction with a cluster comprising thousands of individual bees. Such biohybrid societies open the door to investigation of collective behaviors that necessitate observing and interacting with the animals within a complete social context, as well as for potential applications in augmenting the survivability of these pollinators crucial to our ecosystems and our food supply. This is the author’s version of the work. It is posted here by permission of the AAAS for personal use, not for redistribution. The definitive version was published in Science Robotics, Vol. 8, 76, Mar 2023, DOI: 10.1126/scirobotics.add7385 https://doi.org/10.1126/scirobotics.add7385 

Programmable matter can change its shape, stiffness or other physical properties upon command. Previous work has shown contactless optically controlled matter or magnetic actuation, but the former is limited in strength and the latter in spatial resolution. Here, we show an unprecedented level of control combining light patterns and magnetic fields. A mixture of thermoplastic and ferromagnetic powder is heated up at specific locations that become malleable and are attracted by magnetic fields. These heated areas solidify on cool down, and the process can be repeated. We show complex control of 3D slabs, 2D sheets, and 1D filaments with applications in tactile displays and object manipulation. Due to the low transition temperature and the possibility of using microwave heating, the compound can be manipulated in air, water, or inside biological tissue having the potential to revolutionize biomedical devices, robotics or display technologies. This research is funded by the EU Horizon 2020 research and innovation programme under grant agreement No. 101017746 TOUCHLESS