Challenge Overview
Project Background
The project is a mobile app with augmented reality experience that engages and informs consumers on the vehicle protection products available with their vehicle purchase.
Challenge Overview
Welcome to the “Juno - Augmented Reality Application Challenge”. In this challenge, we are looking to enhance an existing iOS application with augmented reality features.
The application already supports rendering a detailed 3D car model on detected surface and has most of the screens implemented.
The main purpose of this challenge is to extended the capabilities of the app by detecting real cars and displaying the same information as on virtual model.
Requirements
- Implement detecting of real cars with approximate placement of features (the bare minimum is position + direction)
- Display car parts protection tooltips similar to virtual car experience, when selected follow same flow as per virtual car. I.e. show the information popup and selected car parts (where similar part is displayed in 3D model)
- The following screens are in scope:
- 02-01, 02-02 - car detection (outline in the middle can be used just as suggestion to user to make sure car isn’t too far or too close and camera center is pointed towards it, it doesn’t have to match the model or exact position of the car)
- 05-01 - 06-03 - UX similar to virtual car (since we don’t have exact models for every car placement doesn’t need to be perfect and you are not required to show outline, you can also ignore occlusion of the detected car over the AR content, it can be placed on top of video feed)
- 09-02 - App settings, implement Show Hints toggle functionality (disables the tooltips appearing), for the list of states and available products use mock JSON data
- 13-04 - 3D model car type selection popup, implement the selection reflected in side menu, but in the result it may load the same model we currently have
- 14-01, 14-02 - login form and logged out state - imitate login when entering configured in JSON credentials. New experience should return to the landing selection page
- 01-04, ignore the Load Virtual Car/Experience in a Real Car buttons on the side menu.
Suggested route for car detection is to utilize ARKit object tracking which can be trained (see Helpful Resources), once car is detected and an ARReference is available you can then place a hidden virtual 3D model on top of it to display partial features and all the tooltips along.
NOTE that this is a suggestion and actual implementation may utilize a different approach.
Another possible option would be utilizing CoreML and Vision frameworks, however the detected object will lack a 3D position reference and thus might be hard to utilize.
- Existing application source code will be provided in the forum
- Design source will be provided in the forum
Helpful resources
- Creating and training reference objects for ARKit https://developer.apple.com/documentation/arkit/scanning_and_detecting_3d_objects
- List of useful ARKit examples https://github.com/olucurious/Awesome-ARKit
- List of CoreML models https://github.com/likedan/Awesome-CoreML-Models
Technology Overview
- Swift
- ARKit
- iOS 12.0+
Final Submission Guidelines
- Updated source code
- Deploymet and Validation Guides (can be same document)
- Video link demonstrating new functionality