RealityKit & ARKit in SwiftUI

RealityKit framework is developed by Apple for building augmented reality (AR) experiences for iOS, iPadOS and VisionOS devices. It was introduced in 2019 alongside ARKit 3. ARKit provides the underlying technology for AR applications. RealityKit simplifies the development process for AR by offering a high-level API that abstracts away many of the complexities involved in creating AR experiences.

We will start exploring RealityKit & ARKit in SwiftUI. You will need a real device to build & run AR apps. RealityKit works on iOS 13.0+, iPadOS 13.0+, macOS 10.15+, Mac Catalyst 13.0+, and visionOS 1.0+.

Start by creating a new Augmented Reality App project in Xcode.

Set project name, team, org identifier, interface(SwiftUI), language(Swift), content technology(RealityKit)

Click next to create the project.

Xcode creates some boilerplate code which we will delete and create everything from scratch.

import SwiftUI
import RealityKit

struct ContentView : View {
    var body: some View {
        ARViewContainer().edgesIgnoringSafeArea(.all)
    }
}

struct ARViewContainer: UIViewRepresentable {
    
    func makeUIView(context: Context) -> ARView {
       
    }
    
    func updateUIView(_ uiView: ARView, context: Context) {}
    
}

#Preview {
    ContentView()
}

Here, content view calls the ARViewContainer struct which conforms to the UIViewRepresentable protocol which is used to create and manage an ARView instance. UIViewRepresentable is a protocol that allows SwiftUI to interoperate with UIKit views. ARViewContainer acts as a bridge between the SwiftUI framework and the RealityKit framework, allowing us to create 3D AR experiences in our SwiftUI apps.

The makeUIView function is a required method of the UIViewRepresentable protocol. It's called when the SwiftUI view is being created for the first time. In this method, we will create and configure our ARView instance, create or load any 3D models or assets, and add them to the scene. Finally, we will return the created ARView instance.

The updateUIView is another required function of the UIViewRepresentable protocol. It is called whenever the SwiftUI view needs to be updated based on changes to its state or environment. In this method, we will update the ARView instance to reflect any changes in the state of our SwiftUI view.

Let’s start by adding an ARView inside the makeUIView function.

struct ARViewContainer: UIViewRepresentable {
    
    func makeUIView(context: Context) -> ARView {
        
        let arView = ARView(frame: .zero)

We use an ARView instance to display rendered 3D graphics to the user.

Next, we will create AnchorEntity. Anchor entities are used to control how RealityKit places virtual objects into the scene.

struct ARViewContainer: UIViewRepresentable {
    
    func makeUIView(context: Context) -> ARView {
        
        let arView = ARView(frame: .zero)
        let anchor = AnchorEntity(plane: .horizontal)

We will create a purple cube so let’s first create a material object to set the color for the cube.

struct ARViewContainer: UIViewRepresentable {
    
    func makeUIView(context: Context) -> ARView {
        
        let arView = ARView(frame: .zero)
        let anchor = AnchorEntity(plane: .horizontal)
        let material = SimpleMaterial(color: .purple, isMetallic: true)

We are now ready to create a cube, we will use ModelEntiry type to generate a box of size 0.3 meters.

struct ARViewContainer: UIViewRepresentable {
    
    func makeUIView(context: Context) -> ARView {
        
        let arView = ARView(frame: .zero)

        let anchor = AnchorEntity(
            plane: .horizontal
        )

        let material = SimpleMaterial(
            color: .purple,
            isMetallic: true
        )

        let box = ModelEntity(
            mesh: .generateBox(
                size: 0.2
            ),
            materials: [material]
        )

Let’s add box to the anchor and add the anchor to arView’s scene object.

struct ARViewContainer: UIViewRepresentable {
    
    func makeUIView(context: Context) -> ARView {
        
        let arView = ARView(frame: .zero)
        // 1
        let anchor = AnchorEntity(
            plane: .horizontal
        )
        // 2
        let material = SimpleMaterial(
            color: .purple,
            isMetallic: true
        )
        // 3
        let box = ModelEntity(
            mesh: .generateBox(
                size: 0.2
            ),
            materials: [material]
        )
        // 4
        anchor.addChild(box)
        arView.scene.anchors.append(anchor)
        return arView
        
    }
    
    func updateUIView(_ uiView: ARView, context: Context) {}
    
}

Select physical device from the build target and run the app.

Congratulations, you just built your first AR app.