by Andy Jeong, Yue Wang, Professor Mili Shah (Advisor)

Abstract

An emerging technology, Microsoft's Kinect is a motion sensor and video & depth camera to sense interactions and make touchless gestures, body movements and motion capture recordings possible on a Windows PC or Mac for almost no cost. Microsoft have released three versions of Kinect. KinectA is an application for motion tracking via Kinect Sensor – including hand, skeleton and object tracking. This software has been developed for the needs of media artists and designers. It provides a clearly arranged interface with multiple functionalities. The tracked information can be send via OSC to other hard- and software.

Body joint estimation from a single vision system poses limitations in the cases of occlusion and illumination changes, while current motion capture (MOCAP) systems may be expensive. This synchronized Azure Kinect-based MOCAP system serves as a remedy to both of these problems by creating a low-cost, portable, accurate body-tracking system.

Keywords: Motion capture (MOCAP) system, synchronization, Kinect, body-tracking

See Azure.com/Kinect for device info and available documentation.

Link to submitted poster to ACM SIGGRAPH’20: Poster

Link to submitted abstract to ACM SIGGRAPH’20: Abstract

(Received 4 feedback; 3 neutral, 1 slight negative)

Link to documentation: Documentation

Demo

Check out the outcomes on some various movements!Note: this demo experiences some offset due to a parallax problem (devices are at a lower height than the human).

Overview

Flowchart

System Setup

Hardware

• Ubuntu 18.04 / Windows PC machine with USB3.0+ support
• USB Hub for multi-device connection (Targus 4-Port USB 3.0): Link
• USB 3.0 Extension Cable for multi-device connection: Link
• Audio Jack Cables for multi-device connection: Link

Software

• Azure Kinect Senor SDK (K4A) (-lk4a)
• Azure Kinect Body Tracking SDK (K4ABT) (-lk4abt)
• OpenCV (`pkg-config --cflags --libs opencv`)

Building

Test Setup

Configuration

• Daisy-chain configuration: supports connection of 2+ subordinate mode devices to a single master mode device (RS-232)

Testing Environment

Camera Calibration to capture synchronous images

• reference: green screen example from Azure Kinect SDK examples on its GitHub repository

Outcomes

With multiple devices in place, joint estimation is still performed as if there is no occlusion or lighting effect.The following videos and images are tested in the test setup shown above.

Videos Samples

Synchronization

• on the right: joint angles for angles designated as below

Occlusion / Illumination Effect Verification with 3-Device System

Example of selection of data streams by confidence levels per joint

Azure Kinect SDK Details

Azure Kinect SDK is a cross platform (Linux and Windows) user mode SDK to read data from your Azure Kinect device.

The Azure Kinect SDK enables you to get the most out of your Azure Kinect camera. Features include:

• Depth camera access
• RGB camera access and control (e.g. exposure and white balance)
• Motion sensor (gyroscope and accelerometer) access
• Synchronized Depth-RGB camera streaming with configurable delay between cameras
• External device synchronization control with configurable delay offset between devices
• Camera frame meta-data access for image resolution, timestamp and temperature
• Device calibration data access

Current Work

1. Gait Analysis on Exoskeletons

OpenPose, AlphPose, Kinect, Vicon MOCAP system

2. Graphical Visualization of Tracked Body Joints

Media art collaboration

3. Drone Movement Synchronzation from Human Pose

Control of drone system (crazyflie)

Kinect Mocap Plug-in

With this Kinect motion capturing tool, now you can use your own body to control virtual actors in real-time. Now you can capture your motion closer to the camera even when sitting in front of the desk. Thanks to the near mode, your motion data can be captured as close as 50cm to the sensor with full accuracy and precision.

• Publisher: Reallusion Inc.
• Home page:www.reallusion.com
• Last updated: January 10th, 2017

Kinect BVH Mocap

Kinect BVH Mocap is a simple Windows application that captures 3D motion data in BioVision format (.bvh) files using a Kinect sensor. These can be easily imported into Credo Interactive's DanceForms 2.0 choreography and animation software. The application uses the OpenNI/NITE framework for user and skeleton tracking and converts the 3D skeleton point positions.

• Publisher: Integrate Systems Engineering Ltd
• Last updated: October 28th, 2011

iPi Mocap Studio

iPi Mocap Studio is a program designed for tracking an actor's motion by analyzing multi-camera or depth sensor video recordings. As an output, iPi Mocap Studio produces a skeleton animation, that then can be applied to an arbitrary target humanlike character. The app includes features for tuning the final animation, such as jitter removal and trajectory filtering.

• Publisher: iPi Soft
• Home page:docs.ipisoft.com
• Last updated: October 13th, 2015

Axis Neuron

AXIS Neuron is a program designed to manage and calibrate your Perception Neuron system as well as perform basic motion capture. One of the most important features of the program is the ability to stream your BVH data stream as well as export your files to FBX for use in the most popular 3D software programs.

• Publisher: NOITOM
• Last updated: November 29th, 2017

Mocap Device Plug-in

With this Kinect motion capturing tool, now you can use your own body to control virtual actors, in real-time, with motion smoothing optimizations.You can lightspeed your animations and create authentic motions without having to rent an entire mocap studio.

• Publisher: Reallusion Inc.
• Home page:www.reallusion.com
• Last updated: November 21st, 2012

Kinect 3D Photo Capture Tool

The Kinect was planned as a fancy, but just fun, XBox 360 accessory for controlling video games. It is able to connect to a computer and do awesome stuff with it. Controlling robots, scanning rooms, playing air guitars with real sound, drawing with physics and much, much more are just some of the things Kinect have been able to do in a short amount of time.

• Publisher: KinectHacks
• Last updated: March 28th, 2011

Open Exhibits TUIO Kinect

Open Exhibits TUIO Kinect is a simplified standalone installation that captures and tracks Microsoft Kinect depth information and publishes them as TUIO cursor events.This bridge provides an interface to Kinect depth interface for the ability to build and use multitouch applications across a broader range of Windows operating systems, including Windows Vista and Windows XP.

• Publisher: Open Exhibits
• Last updated: November 16th, 2011

Kinect Magic Cursor

To simulate a left mouse button click you just raise your left hand above your shoulder. It certainly could have plenty of uses, an interactive presentation projected onto a whiteboard or wall, which could then be controlled by a user standing in front of it controlling it with their arms is one use and I’m sure you can think of many other uses.

• Publisher: RKC
• Home page:drenton72.wordpress.com
• Last updated: July 28th, 2016

Technitium MAC Address Changer

This software allows you to change (spoof) Media Access Control (MAC) Address of your Network Interface Card (NIC) or Wireless Network Card (WiFi).This software allows you to change IP Address, Gateway, DNS Servers, IPv6 support, enable/disable DHCP in one click, network configuration presets and many more features.

• Publisher: Shreyas Zare,Suyog Limaye,Technitium
• Home page:www.technitium.com
• Last updated: March 15th, 2012

Kinect Motion Capture Software Mac

Change MAC Address

Change MAC Address is a MAC address spoofing tool. It can be used to view and change the MAC ID of all your network adapters. It is possible to generate the ID of a particular manufacturer or a completely random MAC address. This program can also display other network parameters such as DHCP, IP address, DNS servers, and GUID.

• Publisher: LizardSystems
• Home page:lizardsystems.com
• Last updated: December 3rd, 2020

Colasoft MAC Scanner Free

Colasoft MAC Scanner Free is meant to find your IP and MAC addresses. It is free to use and you can always save your results into a TXT file. You have the ability to rearrange the information according to IP address, Mac address and Host Name. You can also export your scanning results into .txt or .csv format files.

Using Kinect For Motion Capture

• Publisher: Colasoft LLC.
• Home page:www.colasoft.com
• Last updated: September 11th, 2018

Colasoft MAC Scanner

Colasoft MAC Scanner is a utility created for network administrators who wish to easily identify the devices that are being connected to their local subnet. By using this program, they can see which computers connected to the local subnet without their permission.

• Publisher: Colasoft
• Home page:www.colasoft.com
• Last updated: September 4th, 2016

Find MAC Address

Find MAC Address can find and list the MAC addresses of remote computers on your network. You can either specify the range of IP addresses or the name of remote PCs to scan for MAC IDs. This program uses ARP, NetBios, NetAPI, WMI, and SNMP methods to find MAC IDs.

• Publisher: LizardSystems
• Home page:lizardsystems.com
• Last updated: December 3rd, 2020

IP-MAC Scanner

IP-MAC Scanner is a tool that will scan your LAN, finding all IP addresses and their corresponding MAC addresses. This program can identify active IP addresses as well as inactive ones. It will allow you to search the network either by ARP or UDP ports.

• Publisher: AthTek Co., Ltd.
• Home page:www.athtek.com
• Last updated: April 28th, 2010

CC Get MAC Address

CCGetMAC is based on Microsoft Netbios technology. If you want to seach a computer's ethernet mac address, you just need to know his computer name or computer ip. That's to say, CCGetMAC can resolve mac address from IP or computer name. Vice versa, you can find ip address from mac address. CCGetMAC also can be used to Wake On LAN. You can use it wake up any computer on your LAN remotely.

• Publisher: CC Get MAC Address
• Home page:www.youngzsoft.net
• Last updated: May 26th, 2020

Hide My MAC Address

Hide My MAC Address has a straightforward interface that displays your network adapter and your computer's real Media Access Control (MAC) address. One click on the Hide button and the program assigns you a new MAC address. We tried using another program to retrieve our test PC's real MAC address and Hide My MAC efficiently protected our system.

• Publisher: My Privacy Tools, Inc.
• Home page:www.hide-my-ip.com
• Last updated: January 17th, 2010

Colasoft MAC Scanner Pro

Colasoft MAC Scanner is a Windows program designed for scanning IP and MAC addresses. It can automatically detect all subnets according to the IP addresses configured on the multiple NICs of a machine and scan the MAC and IP addresses of defined subnets. The program will display the scan results on the list, including the IP address, MAC address, Host Name, and Manufacture.

• Publisher: Colasoft LLC.
• Home page:www.colasoft.com
• Last updated: December 15th, 2011

EMCO MAC Address Scanner

MAC scanner allows you to retrieve MAC addresses from computers located in the local network. Collected MAC addresses identify every network node uniquely and can be used by various network tools, such as Wake-on-LAN and some security utilities. If you like to use any of these tools, an automatic MAC scanning can save many hours of your work.

• Publisher: EMCO Software
• Last updated: November 2nd, 2016

Kinect Motion Capture Software Mac Free