I've been working on AR-related projects for the past year or so. But AR often suffers from the fact that it can only be operated on mobile phone screens. So I want to combine hardware as input to expand the operability of AR.

This is only the initial stage of the project, which can be expanded to include more interesting devices in the future.

M5 ATOM Lite + Lightship ARDK = AR TOY!! Combine hardware and AR to extension the possiblity of virtual world.

Using ATOM Matrix to send & receive BLE data to control AR content.

M5 ATOM Part:

• Extension physical button press
• BLE communication

Lightship ARDK (Unity) Part:

• ARDK Meshing: environment scanning and generat mesh realtime
• Raycast Hit: know where is splat target
• Nature texture painting: convert mesh material as painting result
• BLE communication: Button state

Component Combine

Device BLE connect The BLE <-> Unity communication method is builded by previous artwork AR Fruit Power Gun

Texture painting I get the real time mesh by Lightship ARDK. However, the ARDK meshing does not include uv data. So normal paint on 3d mesh methos isn't working for it. The most difficult thing is how to paint texture on the mesh without any uv data, I need to use custom shader to draw vertex colorthen calculate the texture display result.

The 1st test trying out result is here (try painting on no uv mesh)

Physical device I combine phone, button and M5 Atom into a cardboard case. Make it like splat can, then we could use it to paint AR nature paitning.