Introduction
Sensing is a fundamental requirement in systems dealing with human–computer and machine–machine interactions. The escalating demand for applications such as the Internet of things and digital twins highlight the pressing need to enhance sensing capabilities for systems that collect precise and accurate data [Reference Jones1]. Recent years have witnessed a surge in interest in millimeter-wave (mmWave) technologies, driven by factors such as the deployment of 5G networks and the widespread adoption of mobile computing, transforming our mobile devices into high-performance computers capable of rapid data processing. In the realm of sensing, mmWave technologies offer distinct advantages, including the potential to reduce system component size, achieve sub-millimeter accuracy in detecting spatial changes, and support very high data rate communications [Reference Lynch, Adeyeye, Eid, Hester and Tentzeris2, Reference Kimionis3]. As a result, mmWave systems emerge as a promising avenue for developing the next generation of ubiquitous sensing devices. Simultaneously, the technical advancements of recent years have facilitated the widespread integration of virtual reality (VR) and augmented reality (AR) into various domains, including healthcare, the automotive industry, and the metaverse [Reference Uddin4]. The burgeoning interest in VR and AR underscores the importance of the ability to detect, localize, and determine the orientation of a target for providing an immersive user experience.
Localization, a crucial aspect in VR/AR applications, is commonly performed using radio-frequency identification (RFID) technology, known for its cost-effectiveness and ultra-low-power consumption [Reference Lynch, Adeyeye, Hester and Tentzeris5]. The emergence of mmWave readers allows the extension of RFID technology to higher frequencies through the utilization of millimeter-wave identification (mmID) tags. When operating at mmWave frequencies, the path loss is larger due to higher absorption and greater sensitivity to obstacles such as buildings, resulting in greater signal attenuation [Reference Torres6]. However, by leveraging the bandwidth allocated by the Federal Communications Commission and operating at high frequencies, mmIDs-based systems inherently benefit from increased ranging accuracy and compact, wearable form-factors essential for VR/AR applications. Recent studies, employing a frequency-modulated continuous-wave (FMCW) reader [Reference Lynch, Adeyeye, Hester and Tentzeris5, Reference Strobel, Carlowitz, Wolf, Ellinger and Vossiek7], have demonstrated highly accurate localization estimations. While location detection remains integral in VR/AR, the equally vital ability to detect the precise object orientation is emphasized, defined by the roll, yaw, and pitch axes of rotation in three-dimensional space. Previous works have explored the use of RFID tags for orientation tracking, demonstrating single-axis orientation detection at lower operational frequencies ranging from 1 to 4 GHz [Reference Gupta, Singh, Bal, Kedia and Harish8–Reference Barbot, Rance and Perret10]. However, these systems are intrinsically limited by large size and limited reading range. Machine learning has shown promise in enhancing accuracy, as demonstrated in [Reference Adeyeye, Lynch, Hester and Tentzeris11]. Notably, [Reference Wang, Xu and Xiao12, Reference Akbar, Qi, Alhassoun and Durgin13] showcased the ability to detect orientation estimation in multiple axes using multiple tags, albeit limited by low operational frequency, large form factor, and low detection accuracy.
An earlier version of this paper was presented at the 2023 IEEE 20th European Radar Conference (EuRAD 2023) and was published in its proceedings [Reference Joshi, Lynch, Soto-Valle, Adeyeye, Bahr and Tentzeris14]. Within that work, the authors presented a machine learning enhanced mmID system enabling highly accurate, 
$ \gt 90$%, three-axis orientation prediction at ranges up to 7 m. In this effort, the authors expand on this work by increasing the angular resolution of the system from 10∘ to 5∘ over the entire detection region and performing a dynamic evaluation of the system to highlight its robustness. Additionally, a classification convolutional neural network (CNN) is developed and employed to increase the orientation detection accuracy of the system, thereby addressing the limitations of existing systems.
Proposed low cost mmWave system
Architecture of ultra-low-power Gyro mmID tag
The mmID tag, designed for operation at 24.125 GHz, is comprised of two main sections: the RF front end and the baseband circuit. The RF front end design, similar to the approach presented in [Reference Adeyeye, Lynch, Hester and Tentzeris11], was selected for a three-axes orientation-detecting mmID. This design features four distributed cross-polarized antenna elements, each incorporating a modulating loading constructed with a super low noise amplifier (LNA) field-effect transistor (FET)(CE3520K3 from CEL) and radial stubs. While the tracking of orientation necessitates only three elements, as demonstrated by [Reference Akbar, Qi, Alhassoun and Durgin13], a fourth antenna was integrated for enhanced reliability and to facilitate finer amplitude encoding along the roll axis. Figure 1(a) illustrates the tag layout, where elements A-D exhibit modulating frequencies of 49 kHz, 69 kHz, 85 kHz, and 110 kHz, respectively. The chosen modulation frequencies are carefully selected to avoid harmonic interference, ensuring they remain below the third harmonic of each other. Furthermore, the reduction in modulation frequency contributes to the mmID tag’s low power consumption. Employing a cross-polarization configuration for each antenna minimizes interference from received signals to the reader, with each antenna designed to possess a polarization offset of 15∘ from one another. This configuration enables the encoding of roll angle of rotation information based on the relative received amplitude of each element, while the relative phase difference between two elements remains minimal during rotation in the roll axis.
(a) Proof-of-concept 24 GHz Gyro mmID tag. (b) Diagram of rotational movements for each axis of the mmID. (c) Block diagram of transmitting and receiving chains of the FMCW radar utilized for the interrogation of the mmID tag.
To validate the operational frequency of the mmID, the normalized gain was captured, which is displayed in Fig. 2. Here it can be seen that tag has good agreement with the expected operational frequency. The design of the mmID is implemented on Rogers RO4350B (ϵr = 3.66, 
$\tan\delta$ = 0.0037), with a thickness of 0.51 mm, and antenna elements spaced at 6.5 mm, corresponding to ≈ 
$\frac{\lambda}{2}$. The baseband circuit generates the modulating signal for each antenna element. In this design, the resistor set voltage-controlled oscillator (VCO) LTC6906 is employed for its ability to control the frequency of the generated signal, as well as its low power consumption, ≈ 14.4 µW. To ensure consistent performance of the mmID, a 1.8 V voltage regulator is employed to provide stable voltage to the VCO. Powering of the system is achieved through the utilization of a 3 V coin cell battery, which contributes to the portability of the system.
Measured normalized gain vs frequency of the cross-polarized mmID.
Proof-of-concept FMCW radar system
The proof-of-concept (PoC) reader for the three-axis orientation detection Gyro tag was the Analog Devices EV-RADAR-MMIC2 Evaluation Board, operating at 24 GHz and utilizes FMCW radar technology. This evaluation board integrates the ADF5901 24 GHz monolithic microwave integrated circuit (MMIC) transmitter, ADF5904 24 GHz MMIC receiver, and the ADF4159 13 GHz phase-locked loop. Figure 1(c) shows the block diagram breakdown of the PoC reader. For both the transmitter and receiver, A-INFO LB-180400-20-C-KF horn antennas with 20 dBi gain were employed in a cross-polarized configuration. To enhance system sensitivity, a 40 dB LNA with a noise figure of 3.2 dB was integrated into the receiving antenna. The radar system was configured with a triangular chirp waveform, featuring a frequency slope of 400 MHz/µs and a chirp period of 5 ms. The complete chirp parameters of the PoC FMCW radar system can be found in Table 1.
Chirp parameters of PoC FMCW radar
Signal processing framework
Extraction of Gyro mmID amplitude and phase response
Figure 3 a shows the block diagram of the proposed signal processing scheme. Utilizing the received signal for each corresponding chirp signal sent from the FMCW radar, a range fast Fourier transform (FFT) is applied. Commonly used in scenarios involving FMCW signals, the range FFT is an essential signal processing technique, as it allows for the precise extraction of a targets range and motion information [Reference Lynch, Adeyeye, Eid, Hester and Tentzeris2]. This method works by transforming time-domain radar signals into the frequency domain through the application of the FFT algorithm. The resulting frequency spectrum provides a detailed representation of spectral characteristics, which can then be used in the identification of the targets distance. Peaks in the frequency spectrum correspond to distinct target distances, enhancing the spatial understanding of radar return signal. A sample spectrum is depicted in Fig. 4. This spectrum enables the extraction of modulating beat frequencies for each element on the tag. The negative and positive modulating peaks of each element can be observed, centered around their respective modulation frequencies.
Signal processing chain to extract amplitude and phase response of the tag and the classification CNN neural network used to predict the orientation angle of the mmID tag.
Range-FFT spectrum of the proof-of-concept mmID at broadside with the response of tag elements A-D highlighted.
After executing the range FFT, a custom peak detection algorithm is used to recognize the modulating beat frequencies associated with each element. Subsequently, the phase differences between elements B-A, C-B, D-C, and A-D can be computed. The significance of phase differences in received signals becomes particularly pronounced at extended reading distances. This is attributed to the diminishing signal strength of each element as the range from the reader increases, which results in a reduction in the dynamic range of the amplitude response, with respect to rotation [Reference Hekimian-Williams, Grant, Liu, Zhang and Kumar15]. Arctangent demodulation is a signal processing technique crucial in communication systems for extracting information encoded in modulated signals. The process involves taking the arctangent of the ratio between the in-phase and quadrature components of the modulated signal [Reference Park, Boric-Lubecke and Lubecke16]. The in-phase and quadrature components represent the amplitude and phase information of the signal, respectively. By applying arctangent demodulation, phase information is unwraped, which leads to the extraction of the modulating signal. This demodulation technique is effective in scenarios where accurate phase information retrieval is essential, as it helps mitigate the impact of phase wrapping that can occur in conventional phase demodulation methods. Arctangent demodulation enhances the fidelity of signal demodulation, contributing to the recovery of information transmitted through frequency modulated radar signals.
Machine learning
In this study, a comprehensive performance evaluation was conducted for two machine learning algorithms, K-nearest neighbors (KNN) and CNN. KNN is a popular machine learning algorithm, commonly employed for classification and regression tasks. Operating on the principle of proximity, KNN makes predictions based on the majority class or average of the K-nearest data points in the feature space [Reference Goodfellow, Bengio and Courville17]. While KNN is versatile and widely applicable, its performance is influenced by the choice of the ‘K’ value, determining the number of neighbors considered.
Classification CNNs are a specialized deep learning architecture designed explicitly for the task of classifying input data into distinct categories. Typically used in image classification scenarios, these networks are adept at learning hierarchical representations of features within the data [Reference Goodfellow, Bengio and Courville17]. Comprised of convolutional layers for feature extraction, pooling layers for spatial down-sampling, and fully connected layers for classification, these networks automatically learn and discern intricate patterns, which are crucial for accurate classification. The convolutional layers utilize filters to convolve over input data, capturing local and global features, while pooling layers enhance computational efficiency by reducing spatial dimensions. The final fully connected layers employ learned features to make predictions and assign input data to predefined classes. This training process involves optimizing network parameters through backpropagation and gradient descent, ensuring the model generalizes well to unseen data. To determine the optimal hyper-parameters for the classification CNN, a random-search based optimization technique was applied [Reference Bergstra and Bengio18]. The designed network is made up of a two hidden layers, each containing 10 neurons, with a rectified linear activation and softmax activation functions applied to layers 1 and 2, respectively.
Each model utilizes the phase difference from neighboring antennas, i.e. elements B-A, C-B, D-C, and A-D, along with the amplitude response from each element with respect to orientation as inputs. To standardize the dataset, each feature was scaled using min-max normalization. Additional information on the dataset used for these two models are discussed in more detail below in “Experimental validation of proposed system” section.
Experimental validation of proposed system
Measurement setup
The experimental setup, illustrated in Fig. 5, was designed to enable complete evaluation of the tag’s behavior across different orientations. To enable rotation along all three axes, the tag was affixed to a three-axis gimbal holder, with individual control exerted on each axis facilitated by planetary geared stepper motors, allowing for precise angular steps of 5∘. The motors traversed a range of ±90∘, resulting in 50,653 diverse orientations for the tag. This carefully chosen range ensures complete coverage within the angular space conducive to tag detection.
Experimental setup of the system at a distance of 10 m.
For the visualization and extraction of the transmitted and received signals, the Tektronix DPO 7354 Oscilloscope was employed. Operating at a sampling rate of 500 kHz, the oscilloscope recorded 19 up and down ramps for each angular configuration, totaling 38 observations and yielding 1,924,814 unique samples. Furthermore, a synchronized clock is used to ensure precise alignment between the radar and the oscilloscope, which is crucial for timing purposes to acquire the relative phase differentials between elements accurately. The extracted signals were then exported to MATLAB to allow for post-processing. The experimental parameters where specifically chosen to provide an accurate and robust understanding of the tag’s performance across diverse orientations.
Analysis of machine learning models
Experiments were conducted at 11 distinct distances (0.5 m, 1 m, 2 m, 3 m, 4 m, 5 m, 6 m, 7 m, 8 m, 9 m, and 10 m), forming a comprehensive dataset of 21,172,954 samples by aggregating data from each experiment. Given the dataset’s substantial size, an 80/20 train-test split was employed, with 80% of the data utilized for model training and the remaining 20% for assessing its performance.
The results, presented in Table 2, reveal that while the KNN demonstrated comparable accuracy to the CNN at distances up to 3 m, its accuracy significantly declined at longer distances. Conversely, the CNN achieved high accuracy (
$ \gt 90$%) at distances up to 8 m. While the KNN demonstrated acceptable results in [Reference Joshi, Lynch, Soto-Valle, Adeyeye, Bahr and Tentzeris14], this study faced challenges in replicating similar accuracy. The elevation of angular resolution from 10∘ to 5∘ resulted in increased similarity in amplitude and phase for each of the four antennas during tag rotation. Consequently, data for each angular configuration grouped closely together, which hinders the algorithm’s ability to distinguish different orientation configurations. Tailored for advanced feature extraction, the CNN excelled in capturing intricate patterns and spatial relationships in the phase difference and amplitude response of four antennas, contributing to their excellent performance in this study.
Comparison of accuracy using K-nearest neighbors and classification CNN models
In Fig. 6, the confusion matrices for all three axes at a distance of 5 m are illustrated using the classification CNN. Each matrix is a composite representation of rotations, where one axis is held constant at a specific angle, and the remaining two axes rotate within ± 90∘. For instance, the yaw confusion matrix at 90∘ encompasses all rotations of pitch and roll while maintaining the yaw axis at 90∘. Here, it can be seen that the model was able to achieve high accuracy along each axis. Additionally, this measurement was able to achieve a true positive rate of 93%, highlighting the models ability to correctly predict the orientation of the mmID. Notably, a discernible pattern emerges as false estimations manifest consistently when the angle exceeds ± 80∘ across all three axes. This consistent trend is observable in other experiments and is likely attributed to the tag exceeding the beamwidth of the transmitting and receiving antennas, leading to the diminished accuracy.
Confusion matrices at 5 m: (a) roll axis fixed with yaw and pitch axes rotating, (b) yaw axis fixed with roll and pitch axes rotating, (c) pitch axis fixed with roll and yaw axes rotating.
Dynamic system evaluation
To evaluate the system’s robustness, a dynamic evaluation was conducted involving a sequence of experiments where the mmID traversed a predetermined path, as seen in Fig. 7. With the tag placed at a range of 5 m, seven experiments were executed, encompassing a range of rotation speeds from 0.25× to 1.75×, equivalent to 3.5–23.5∘/sec, relative to the tag’s rotation speed used during the training of both machine learning models. The detailed results of these experiments are outlined in Table 3. An analysis of the results indicate that, although the accuracy of the CNN experiences a decline with increasing speed, it consistently maintains a high accuracy rate exceeding 92%. Additionally, the CNN model consistently outperforms the KNN model, consistent with the previously presented findings. These experiments not only highlights the system’s adaptability to dynamic conditions but also emphasizes the CNN model’s effectiveness in sustaining accurate orientation tracking, even under varying rotational dynamics.
Programmed path of the mmID for system evaluation.
Comparison of accuracy using K-nearest neighbors and classification CNN for varying rotation speeds
Conclusion
In summary, the authors introduced a highly scalable, ultra-low power 24 GHz mmID tag for predicting its three-axis orientation at extended ranges by employing a classification CNN machine learning algorithm. The proposed system exhibited an accuracy exceeding 90% even at a range of 8 m from the PoC reader. An evaluation of the system was performed at various rotation speeds, in which the system was able to maintain high accuracy (
$ \gt 90$%) throughout each experiment. Further evaluation of the proposed system will be explored including the dynamic orientation tracking of the mmID integrated with a moving target. In the context of VR and AR applications demanding low latency and devices with highly accurate orientation tracking, this cost-effective mmID tag emerges as a promising solution. Moreover, the system presents a unique opportunity for drone swarms applications, with the integration of multiple tags. With capabilities extending to immersive experiences and the mapping of AR objects into user interactions, this mmID tag stands out as a viable candidate for future systems. Additionally, the proposed PoC mmID sensor holds potential for advancing wireless motion capture systems. By integrating multiple mmID tags to provide accurate location and orientation information, the system offers a cost-effective alternative to traditional optical camera configurations, contributing to increased accessibility and efficiency in motion capture technology.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S1759078724000965.
Competing interests
The author(s) declare none.
Marvin Joshi received the B.S. and M.S. degree in electrical engineering from the University of Tennessee, Knoxville, TN, USA in 2020 and 2022, respectively. He is currently pursuing the Ph.D. degree in electrical engineering at the Georgia Institute of Technology, Atlanta, GA, USA. He is currently a Research Assistant with the ATHENA Group, Georgia Institute of Technology. His current research interest focuses on the integration of RADAR and artificial intelligence/machine learning with RF modules, specifically mmIDs for localization and tracking applications.
Charles A. Lynch III received the B.S. degree in electrical engineering from Rose-Hulman Institute of Technology, Terre Haute, IN, USA, in 2019. He is currently pursuing the M.S. and Ph.D. degrees in electrical engineering at the Georgia Institute of Technology, Atlanta, GA, USA. He is currently a Research Assistant with the ATHENA Group, Georgia Institute of Technology. His current research interest focuses on the design, simulation, and fabrication of RF and millimeterwave devices, specifically RFIDs and mmIDs and combining these low-power, wearable, ultralow-cost devices with radar for localized sensing in the Internet of things systems. Mr. Lynch received the audience choice in the 3MT Competition at the 2023 IEEE IMS conference.
Ajibayo Adeyeye received his B.S. degree in electrical and electronics engineering magna cum laude from the Rose-Hulman Institute of Technology, Terre Haute, IN, USA. He went on to complete both his Master’s and Ph.D. degrees in electrical engineering at the Georgia Institute of Technology, Atlanta, GA, USA, in 2022. His research interests are centered around the integration of RADAR and RFID systems for localization applications. Adeyeye has been recognized for his contributions to the field, securing first place in the 2019 IEEE Microwave Theory and Techniques Society International Microwave Symposium Student Design Competition on Backscatter Radio. He was also a Best Student Paper finalist at the 2021 International Microwave Symposium.
Genaro Soto-Valle received the B.S. degree in nanotechnology from the National Autonomous University of Mexico, in 2020. He is currently pursuing the M.S. and Ph.D. degrees in electrical engineering at the Georgia Institute of Technology, in Atlanta, GA. As a Research Assistant in the ATHENA group, his research interests focus on additively manufactured RF systems for wireless environmental sensing and localization, particularly by using mmWave modules targeted for Internet of things systems. More recently, he has been working on novel additive manufacturing techniques to enable micrometer-scale modules for sub-THz applications.
Manos M. Tentzeris (S’89-M’92-SM’03-F’10) received the Diploma degree (magna cum laude) in electrical and computer engineering from the National Technical University of Athens, Athens, Greece, and the M.S. and Ph.D. degrees in electrical engineering and computer science from the University of Michigan, Ann Arbor, MI, USA. He is currently Ed and Pat Joy Chair Professor with the School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA, USA,where he headsthe ATHENA Research Group (20 researchers). He has served as the Head of the GT ECE Electromagnetics Technical Interest Group, as the Georgia Electronic Design Center Associate Director of RFID/Sensors research, as the Georgia Institute of Technology NSF-Packaging Research Center Associate Director of RF Research, and as the RF Alliance Leader. He has helped develop academic programs in 3D/inkjet-printed RF electronics and modules, flexible electronics, origami and morphing electromagnetics, highly integrated/multilayer packaging for RF, millimeter-wave, sub-THz and wireless applications using ceramic and organic flexible materials, paper-based RFID’s and sensors, wireless sensors and biosensors, wearable electronics, “Green” and transient electronics, energy harvesting and wireless power transfer, nanotechnology applications in RF, microwave MEMs, and SOP-integrated (UWB, multiband, mmW, and conformal) antennas. He has authored more than 850 papers in refereed journals and conference proceedings, 7 books, and 26 book chapters. He was a Visiting Professor with the Technical University of Munich, Munich, Germany, in 2002, with GTRI-Ireland, Athlone, Ireland, in 2009 and with LAAS-CNRS, Toulouse, France, in 2010 and a Humboldt Guest Professor with FAU, Nuremberg, Germany in 2019.
Dr. Tentzeris was a recipient/co-recipient of the 2024 Georgia Tech Outstanding Achievement in Research Innovation Award, 2023 Proceedings of IEEE Best Paper Award, 2022 Georgia Tech Outstanding Doctoral Thesis Advisor Award, the 2021 IEEE Antennas and Propagation Symposium (APS) Best Student Paper Award, the 2019 Humboldt Research Prize, the 2017 Georgia Institute of Technology Outstanding Achievement in Research Program Development Award, the 2016 Bell Labs Award Competition 3rd Prize, the 2015 IET Microwaves, Antennas, and Propagation Premium Award, the 2014 Georgia Institute of Technology ECE Distinguished Faculty Achievement Award, the 2014 IEEE RFID-TA Best Student Paper Award, the 2013 IET Microwaves, Antennas and Propagation Premium Award, the 2012 FiDiPro Award in Finland, the iCMG Architecture Award of Excellence, the 2010 IEEE Antennas and Propagation Society Piergiorgio L. E. Uslenghi Letters Prize Paper Award, the 2011 International Workshop on Structural Health Monitoring Best Student Paper Award, the 2010 Georgia Institute of Technology Senior Faculty Outstanding Undergraduate Research Mentor Award, the 2009 IEEE TRANSACTIONS ON COMPONENTS AND PACKAGING TECHNOLOGIES Best Paper Award, the 2009 E. T. S. Walton Award from the Irish Science Foundation, the 2007 IEEE AP-S Symposium Best Student Paper Award, the 2007 IEEE MTT-S IMS Third Best Student Paper Award, the 2007 ISAP 2007 Poster Presentation Award, the 2006 IEEE MTT-S Outstanding Young Engineer Award, the 2006 Asia Pacific Microwave Conference Award, the 2004 IEEE TRANSACTIONS ON ADVANCED PACKAGING Commendable Paper Award, the 2003 NASA Godfrey “Art” Anzic Collaborative Distinguished Publication Award, the 2003 IBC International Educator of the Year Award, the 2003 IEEE CPMT Outstanding Young Engineer Award, the 2002 International Conference on Microwave and Millimeter-Wave Technology Best Paper Award (Beijing, China), the 2002 Georgia Institute of Technology–ECE Outstanding Junior Faculty Award, the 2001 ACES Conference Best Paper Award, the 2000 NSF CAREER Award, and the 1997 Best Paper Award of the International Hybrid Microelectronics and Packaging Society. He was the General Co-Chair of the 2023 IEEE Wireless Power Transfer Technology Conference and Expo (WPTCE) in San Diego and of the 2019 IEEE APS Symposium in Atlanta. He was the TPC Chair of the IEEE MTT-S IMS 2008 Symposium and the Chair of the 2005 IEEE CEM-TD Workshop. He is the Vice-Chair of the RF Technical Committee (TC16) of the IEEE CPMT Society. He is the Founder and Chair of the RFID Technical Committee (TC24) of the IEEE MTT-S and the Secretary/Treasurer of the IEEE C-RFID. He has served as an Associate Editor of the IEEE TRANSACTIONS ON MICROWAVE THEORY AND TECHNIQUES, the IEEE TRANSACTIONS ON ADVANCED PACKAGING, and the International Journal on Antennas and Propagation. He has given more than 150 invited talks to various universities and companies all over the world. He is a member of the URSI-Commission D and the MTT-15 Committee, an Associate Member of EuMA, a Fellow of the Electromagnetic Academy, and a member of the Technical Chamber of Greece. He is currently one IEEE EPS Distinguished Lecturer and he has served as one of the IEEE MTT-S Distinguished Microwave Lecturers and as one of the IEEE CRFID Distinguished Lecturers.
Open access
