This paper tackles the task of hand-object contact transition detection in egocentric video, which aims to localize the two temporal boundaries of a contact episode: the moment a hand first establishes contact with an object (touch) and the moment contact is released (release). We propose a unified framework that jointly models the interacting hand and object to accurately localize both transition types. The proposed framework augments hand-centric spatiotemporal representations with object-aware features learned directly from the video and introduces a transition-aware event spotting formulation tailored to contact transitions, without relying on external object or hand-object interaction detectors during inference. To support this task, we present ContactTransition, a dataset comprising 4,021 egocentric videos with 8,498 touch and 8,077 release annotations, together with object-of-interest supervision. Experiments demonstrate that the proposed method consistently outperforms strong two-stage and end-to-end event-spotting baselines on both transition types. Finally, we show that detected contact transitions provide effective temporal anchors for downstream egocentric hand-object reasoning.

Knowing the position of a display screen where information is presented to a viewer is a crucial capability that can drive innovative applications. This paper introduces ScreenLocNet, a novel machine learning architecture for predicting the 3D location of a display screen relative to a camera, even when the screen itself is not visible and without requiring a calibration procedure to establish the screen-camera relationship. ScreenLocNet processes a sequence of video frames showing a viewer naturally interacting with screen content, and leverages patterns in the viewer's gaze behavior to predict the 3D coordinates of the screen's four corners. The architecture features a dual-branch design: a static-scene branch that extracts perspective information of the environment and a gaze-dynamic branch that captures temporal gaze behavior. These branches are integrated through a transformer encoder, enabling the network to combine spatial and temporal cues effectively. To train and evaluate ScreenLocNet, we collected a dataset of 151 videos from 34 subjects across three distinct locations, paired with ground truth screen corner positions to assess cross-scene generalization. Our results demonstrate that ScreenLocNet can estimate screen location within a 15 cm margin of error—sufficiently useful for some downstream applications, such as detecting when a person is not focusing on the displayed content.

We address the challenging task of detecting the precise moment when hands make contact with objects in egocentric videos. This frame-level detection is crucial for augmented reality, human-computer interaction, assistive technologies, and robot learning applications, where contact onset signals action initiation or completion. Temporally precise detection is particularly challenging due to subtle hand motion variations near contact, frequent occlusions, fine-grained manipulation patterns, and the inherent motion dynamics of first-person perspectives. To tackle these challenges, we propose a Hand-informed Context Enhanced module (HiCE; pronounced `high-see`) that leverages spatiotemporal features from hand regions and their surrounding context through cross-attention mechanisms, learning to identify potential contact patterns. Our approach is further refined with a grasp-aware loss and soft label that emphasizes hand pose patterns and movement dynamics characteristic of touch events, enabling the model to distinguish between near-contact and actual contact frames. We also introduce TouchMoment, an egocentric dataset containing 4,021 videos and 8,456 annotated contact moments spanning over one million frames. Experiments on TouchMoment show that, under a strict evaluation criterion that counts a prediction as correct only if it falls within a two-frame tolerance of the ground-truth moment, our method achieves substantial gains and outperforms state-of-the-art event-spotting baselines by 16.91% average precision.

AI language generators are now ubiquitous but typically produce generic text that fails to reflect individual differences. Here, we introduce PsychAdapter, a lightweight LLM architectural modification that uses empirically derived links between language and personality, demographic, and mental health traits to generate trait-reflective text, regardless of prompt. PsychAdapter was applied to GPT-2, Gemma-2B, and LLaMA-3, and expert raters confirmed that the generated text matched the specified traits: it produced Big Five personality traits with 87.3% and depression and life satisfaction with 96.7% accuracy. PsychAdapter is a novel method for embedding psychological behavioral patterns into language models by conditioning every transformer layer, without relying on prompting. Beyond personality-conditioned generation, this approach has potential uses for simulated patients reflecting psychopathology and translation tailored to reading or educational level. It also enables generation of characteristic sentences for studying the language of traits, expanding the language processing toolkit for psychology.

We address the challenging task of identifying, segmenting, and tracking hand-held objects, which is crucial for applications such as human action segmentation and performance evaluation. This task is particularly challenging due to heavy occlusion, rapid motion, and the transitory nature of objects being hand-held, where an object may be held, released, and subsequently picked up again. To tackle these challenges, we have developed a novel transformer-based architecture called HOIST-Former. HOIST-Former is adept at spatially and temporally segmenting hands and objects by iteratively pooling features from each other, ensuring that the processes of identification, segmentation, and tracking of hand-held objects depend on the hands' positions and their contextual appearance. We further refine HOIST-Former with a contact loss that focuses on areas where hands are in contact with objects. Moreover, we also contribute an in-the-wild video dataset called HOIST, which comprises 4,125 videos complete with bounding boxes, segmentation masks, and tracking IDs for hand-held objects. Through experiments on the HOIST dataset and two additional public datasets, we demonstrate the efficacy of HOIST-Former in segmenting and tracking hand-held objects.

The harmonized commodity description and corresponding coding system (HS Code System) created by the World Customs Organization (WCO) are internationally used to classify standard transaction goods from their descriptions. The system uses the four-level hierarchical structure to arrange thousands of different codes. However, in practice, the traditional and manual methods for classifying a large number of items is a labor-consuming work and also prone to error. In order to assist the customs officers as well as many companies, we proposed a deep learning model with self-attention mechanism along side hierarchical classifying layers to improve the accuracy of classification of Vietnamese short text from goods declarations. Experimental results indicated the potential of these approaches with high accuracy.