Augment Mobile SDK - iOS

Getting Started

How to add Augment Player SDK to an existing project?

Import the SDK

  • CocoaPods

Just add this pod to your target :

pod 'AugmentPlayerSDK', '~> 4.0.6'
  • Manually
  1. Download the latest .framework file
  2. Add it to Embedded Binaries section in your project’s General tab in Xcode

Tracker resources

To make the Augment Player SDK fully functional, you also need to download this file, unzip it, copy extracted the files into your project and add them to the Copy Bundle Resources of your app’s target Build Phases.

Overview

Import AugmentPlayerSDK’s umbrella header with the following statement:

// YourApp.h or YourApp-BrindgingHeader.h
@import AugmentPlayerSDK;

You can now use the main 3 following classes:

ProductsDataController :

You can query products via ProductsDataController, it returns you Product instances.

AugmentSDK :

You can add model 3Ds to AugmentPlayer with Product instances from ProductsDataController

AugmentPlayer :

Once you add model 3Ds to AugmentPlayer via AugmentSDK you can resume or pause rendering

That’s all!

A working example

This example covers how to implement a simple UIViewController that uses Augment Player SDK

Setup

You need an instance of AugmentPlayer and an AugmentPlayerView (AGTView in ObjectiveC).

// These are demo credentials linked to the demo catalog below
NSString* APP_ID = @"357fee36746668573ceb2f5957c4869ee1a62a112639bac9b0fae43c7c431692";
NSString* APP_SECRET = @"80ae1420e164e0440d5329067bcdd953e9fa6c63b75c001c06d169a4f11268c5";
NSString* VUFORIA_KEY = @"ATQqCM7/////AAAAGXLs+GRi0UwXh0X+/qQL49dbZGym8kKo+iRtgC95tbJoCWjXXZihDl5pzxoca2JxLcYxBJ2pIeIE4dNcK0etMeb1746L7lq6vSFen43cS7P1P/HXjwHtUouV5Xus2U0F7WHUTKuO629jKFO13fBQczuY52UJcSEhsu9jHPMaupo5CpqQT3TFTQjlhzHhVXiVMEqq7RI+Edwh8TCSfGAbNRdbIELTfK+8YDYqwEHDbp62mFrs68YnCEQZDrpcLyC8WzFCVZtnUq3Cj3YBUfQ6gNnENYiuLf06gAAF/FcaF65VYveGRBbp3hpkqolX28bxPiUYNVknCSFXICPHciVntxF+rcHW5rrX7Cg/IVFGdNRF";
// These are demo credentials linked to the demo catalog below
let APP_ID = "357fee36746668573ceb2f5957c4869ee1a62a112639bac9b0fae43c7c431692"
let APP_SECRET = "80ae1420e164e0440d5329067bcdd953e9fa6c63b75c001c06d169a4f11268c5"
let VUFORIA_KEY = "ATQqCM7/////AAAAGXLs+GRi0UwXh0X+/qQL49dbZGym8kKo+iRtgC95tbJoCWjXXZihDl5pzxoca2JxLcYxBJ2pIeIE4dNcK0etMeb1746L7lq6vSFen43cS7P1P/HXjwHtUouV5Xus2U0F7WHUTKuO629jKFO13fBQczuY52UJcSEhsu9jHPMaupo5CpqQT3TFTQjlhzHhVXiVMEqq7RI+Edwh8TCSfGAbNRdbIELTfK+8YDYqwEHDbp62mFrs68YnCEQZDrpcLyC8WzFCVZtnUq3Cj3YBUfQ6gNnENYiuLf06gAAF/FcaF65VYveGRBbp3hpkqolX28bxPiUYNVknCSFXICPHciVntxF+rcHW5rrX7Cg/IVFGdNRF"
@interface ViewController ()

@property (weak, nonatomic) IBOutlet UIActivityIndicatorView* activityIndicator;
@property (nonatomic, weak) NSURLSessionDownloadTask* activeDownloadTask;

// AugmentPlayerSDK part
@property (nonatomic, weak) IBOutlet AGTView* ARView;
@property (nonatomic, strong) id<AGTAugmentPlayer> augmentPlayer;
@property (nonatomic, strong) AGTAugmentSDK* augmentSDK;

@property (nonatomic, weak) id<AGTCancellable> activeNetworkTask;

@end

// ...

- (void)viewDidLoad {
    [super viewDidLoad];

    self.augmentSDK = [AGTAugmentSDK alloc] initWithClientID: APP_ID
                                                clientSecret: APP_SECRET];

    // WARNING: [augmentSDK augmentPlayer] will now create a NEW instance for every call
    self.augmentPlayer = [augmentSDK createAugmentPlayer];

    // AGTView needs to know its renderer. if you do not set its augmentPlayer
    // property, it won't display anything
    [self.ARView setAugmentPlayer:self.augmentPlayer];
    [self prepareARSession];
}

- (void)viewWillDisappear:(BOOL)animated {
    [super viewWillDisappear: animated];
    NSError* error = nil;
    [self.augmentPlayer pause:&error];
    if (error != nil) {
        // Handle error
    } else {
        [self.augmentPlayer unloadAll];
    }
}
class ViewController: UIViewController {

    @IBOutlet weak var activityIndicator: UIActivityIndicatorView!

    // AugmentPlayerSDK part
    @IBOutlet weak var arView: AugmentPlayerView!
    var augmentPlayer: AugmentPlayer!
    var augmentSDK: AugmentSDK!

    var activeNetworkTask: Cancellable!

    override func viewDidLoad() {
        super.viewDidLoad()

        /**
        *  You need these secrets to authenticate within Augment Web API and query
        *  and/or download assets of your products.
        */
        augmentSDK =  AugmentSDK(
            clientID: APP_ID,
            clientSecret: APP_SECRET,
        )

        /**
        *  Create a new AugmentPlayer instance and retain it.
        */
        augmentPlayer = augmentSDK.createAugmentPlayer()

        /**
         *  AugmentPlayerView needs to know its renderer.
         * If you do not set its augmentPlayer property, it won't display anything.
         */
        arView.augmentPlayer = augmentPlayer

        prepareARSession()
    }

    override func viewWillDisappear(_ animated: Bool) {
        super.viewWillDisappear(animated)
        do {
            try augmentPlayer.pause()
            augmentPlayer.unloadAll()
        } catch let error {
            // Handle error
        }
    }
}

Initialize environment

You need to grant camera permission before initializing environment

- (void)prepareARSession {
    // You need to grant camera permission before starting AugmentPlayerSDK
    // otherwise you will have an error during initialization: "Cannot access the camera"
    AVAuthorizationStatus authStatus = [AVCaptureDevice authorizationStatusForMediaType:AVMediaTypeVideo];
    if (authStatus == AVAuthorizationStatusAuthorized) {
        [self loadMyModel];
    }
    else if (authStatus == AVAuthorizationStatusNotDetermined) {
        [AVCaptureDevice requestAccessForMediaType:AVMediaTypeVideo
            completionHandler:^(BOOL granted) {
                if (granted) {
                    dispatch_async(dispatch_get_main_queue(), ^{
                        [self loadMyModel];
                    });
                }
            }];
    }
}
import AVFoundation

func prepareARSession() {
    /**
     * you need to grant camera permission before starting AugmentPlayerSDK
     * otherwise you will have an error during initialization: "Cannot access the camera"
     */
    let authStatus = AVCaptureDevice.authorizationStatus(forMediaType: AVMediaTypeVideo)
    if authStatus == AVAuthorizationStatus.authorized {
        loadMyModel()
    }
    else if authStatus == AVAuthorizationStatus.notDetermined {
        AVCaptureDevice.requestAccess(forMediaType: AVMediaTypeVideo) {
            granted in
            if granted {
                self.loadMyModel()
            }
        }
    }
}

AugmentPlayer tracking status

Now the AugmentPlayer comes with a status tracker to let you know what’s happening internally. Here are the all the possible TrackingStatus:

AGTTrackingStatusInitializing, //The underlying AR system is getting set-up.
AGTTrackingStatusNormal, //The AR tracking is working properly and the discovery mode is active.
AGTTrackingStatusNotAvailable, //The AR tracking is not available; try reseting the player.
AGTTrackingStatusLimitedExcessiveMotion, //Tracking is limited due to a excessive motion of the camera.
AGTTrackingStatusLimitedInsufficientFeatures, //Tracking is limited due to a lack of features (illumination or details) visible to the camera.
AGTTrackingStatusFeaturesDetected, //Some features has been detected. Invite the user to explorer a little more to detected a plane to place the product.
AGTTrackingStatusPlaneDetected, //Your camera is pointing right over a surface. Invite the user to tap to place the product on the plane.
AGTTrackingStatusError, //Something went wrong with the player, this message will tell you what.
AGTTrackingStatusTrackerDetected, //The user has detected a tracker (non-ArKit devices only).
.initializing, //The underlying AR system is getting set-up.
.normal, //The AR tracking is working properly and the discovery mode is active.
.notAvailable, //The AR tracking is not available; try reseting the player.
.limitedExcessiveMotion, //Tracking is limited due to a excessive motion of the camera.
.limitedInsufficientFeatures, //Tracking is limited due to a lack of features (illumination or details) visible to the camera.
.featuresDetected, //Some features has been detected. Invite the user to explorer a little more to detected a plane to place the product.
.planeDetected, //Your camera is pointing right over a surface. Invite the user to tap to place the product on the plane.
.error, //Something went wrong with the player, this message will tell you what.
.trackerDetected, //The user detected a tracker (non-ArKit devices only)

You can implement TrackingStatusDelegate (AGTAugmentPlayerTrackingStatusDelegate in ObjC) and register to AugmentPlayer:

- (void)trackStatus:(AGTTrackingStatus)status withErrorMessage:(NSString*)errorMessage {
    // handle events
}

- (void)viewDidLoad {
    // ....
    self.augmentPlayer.trackingStatusDelegate = self;
}
    public func trackStatus(_ status: TrackingStatus, withErrorMessage errorMessage: String?) {
        // handle events
    }

    override public func viewDidLoad() {
        // ....

        // Register to AugmentPlayer here
        agtPlayer.trackingStatusDelegate = self
    }

Products and catalog

Your first catalog

Augment SDK work with catalogs, when you ask for an API KEY we create a catalog on our server that will link between your references and our models.

To help you test our SDK we provide a demo catalog containing some demo references, here are the data

IdentifierBrandName
81SamsungTV
82LenovoYoga 900
83SamsungWasher
84WhirlpoolFridge
86FINLANDEKCouch
89Restoration HardwareChair
90Restoration HardwareTable
91TargetChair
92TargetOutdoor Chair
93Bed BathTable
94Bed BathCurtains
95Sun LoungerFrontgate
96BallardTable
97AdidasShoe
98NintendoSwitch

EAN are optional, pass an empty string for the demo

Query product

Use ProductsDataController in order to query products over Augment database. If the product exists in Augment database, ProductsDataController returns an object conforming to Product within completion block. You also can check if a query for given identifier had done before via ProductsDataController as seen below:

- (void)loadMyModel {
    id<AGTProduct> cachedProduct = [self.augmentSDK.productsDataController productForIdentifier: @"84"]; // From the catalog
    if (cachedProduct != nil) {
        [self addProductToAugmentPlayer:cachedProduct];
    }
    else {
        [self queryAndAddProductToAugmentPlayerWithIdentifier: @"84"];
    }
}
func loadMyModel() {
    if let cachedProduct = augmentSDK.productsDataController.product(forIdentifier: "84") { // From catalog
        addProductToAugmentPlayer(product: cachedProduct)
    }
    else {
        queryAndAddProductToAugmentPlayer(withIdentifier:"84")
    }
}

Load product

- (void)queryAndAddProductToAugmentPlayerWithIdentifier:(NSString*)productIdentifier {
    // you cannot directly add a product to AugmentPlayer
    // first, you need to check if your product has a model3D in Augment's database
    [self.activityIndicator startAnimating];
    __weak typeof(self) weakSelf = self; //__
    [self.augmentSDK.productsDataController
     checkIfModelDoesExistForProductIdentifier: @"84" brand: @"Whirlpool" name: @"Fridge" ean: nil completion:^(id<AGTProduct> product, NSError* queryError) {
         [weakSelf.activityIndicator stopAnimating];
         if (product != nil && queryError == nil) {
             NSLog(@"ViewController: Product has a model3D in Augment database!");
             [weakSelf addProductToAugmentPlayer:product];
         }
         else {
             // this error might be "model 3D not found in Augment database or anything else
             NSLog(@"ViewController: Query Error %@", queryError.localizedDescription);
         }
     }];
}

- (void)addProductToAugmentPlayer:(id<AGTProduct>)product {
    // after making sure that your product has a model3D, now you can add it to
    // AugmentPlayer via AugmentSDK
    [self.activityIndicator startAnimating];
    __weak typeof(self) weakSelf = self; //__
    [self.augmentSDK
     addProductToAugmentPlayer:product
     downloadProgress:^(NSProgress* progress) {
         NSLog(@"Download progress: %%%f", progress.fractionCompleted);
     }
     operationCompletionWithModelIdentifier:^(NSUUID *itemIdentifier, NSArray<NSError*> * errors) {
         [weakSelf.activityIndicator stopAnimating];
         if (errors.count <= 0 && itemIdentifier != nil) {
             NSLog(@"ViewController: model with id %@ added", itemIdentifier);
             // At that moment you made sure that a model 3D is added to
             // AugmentPlayer and now you can start AugmentPlayer
            [weakSelf startAR];
         }
         else {
             // As addProductToAugment method does more than one thing under the
             // hood, it may return multiple errors
             NSMutableString* errorString = [[NSMutableString alloc] init];
             [errors enumerateObjectsUsingBlock:^(NSError * obj, NSUInteger idx, BOOL * stop) {
                 [errorString appendString:obj.localizedDescription];
                 [errorString appendString:@" "];
             }];
             NSLog(@"ViewController: an error occurred %@", errorString);
         }
     }];
}
func queryAndAddProductToAugmentPlayer(withIdentifier productIdentifier: String) {
    /**
     *  you cannot directly add a product to AugmentPlayer
     *  first, you need to check if your product has a model3D in Augment's database
     */
    activityIndicator.startAnimating()
    augmentSDK.productsDataController.checkIfModelDoesExist(forProductIdentifier: DummyProductIdentifier, brand: DummyProductBrand, name: DummyProductName, ean: DummyProductEAN) {
        product, queryError in
        if let queryError = queryError {
            print("ViewController: Query Error \(queryError.localizedDescription)")
            return
        }

        if let product = product {
            print("ViewController: Product has a model3D in Augment database!")
            self.addProductToAugmentPlayer(product: product)
        }
        else {
            print("Model 3D not found in Augment database")
        }
    }
}

func addProductToAugmentPlayer(product: Product) {
    /**
     *  after making sure that your product has a model3D, now you can add it to
     *  AugmentPlayer via AugmentSDK
     */
    activityIndicator.startAnimating()

    /**
     *  this method will first check your local disk, if you already have
     *  requested model3D assets on your local disk it won't initiate a download
     *  operation and add it to AugmentPlayer right away if local disk check
     *  fails, it will initiate a download operation and returns it as
     *  NSURLSessionDownloadTask
     */
    augmentSDK.addProduct(toAugmentPlayer: product, downloadProgress: {
        (progress: Progress) in
        print("Download progress: \(progress.fractionCompleted)")
    }) {
        itemIdentifier, errors in
        self.activityIndicator.stopAnimating()
        if let errors = errors {
            /**
             * As addProductToAugment method does more than one thing under the
             * hood, it may return multiple errors
             */
            let errorMessage = errors.map { $0.localizedDescription }.joined(separator: ", ")
            print("ViewController: an error occurred \(errorMessage)")
            return
        }

        if let itemIdentifier = itemIdentifier {
            print("ViewController: model with id \(itemIdentifier) added")
            self.startAR()
        }
        else {
            print("Impossible to show this product yet")
        }
    }
}

Visualize

- (void)startAR {
    [self.augmentSDK.augmentPlayer resume];
}
func startAR() {
    augmentSDK.augmentPlayer.resume()
}

After this step, you should be seeing the model on your screen

Re-centering products

It happens that users lose sight of their product while looking away in augmented reality. You can re-center the products at any time by calling:

- (IBAction)recenterProducts:(id)sender {
    [self.augmentSDK.augmentPlayer recenterProducts];
}
@IBAction func recenterProducts(sender: AnyObject) {
    augmentSDK.augmentPlayer.recenterProducts()
}

Detecting Interactions

You can listen and react to the user’s gestures using the SDK protocol AugmentPlayerModelGestureDelegate, which let you know when the model is placed, moved or rotated, and register it to AugmentPlayer.

- (void)onModelAdded:(NSString*)model3DUuid;
- (void)onModelTranslated:(NSString*)model3DUuid;
- (void)onModelRotated:(NSString*)model3DUuid;
func onModelAdded(model3DUuid: String)
func onModelTranslated(model3DUuid: String)
func onModelRotated(model3DUuid: String)

You can also get corresponding Product information (if any) using ProductsDataController method productForModel3DIdentifier: to know which product the user interacted with.

Taking a screenshot

Using takeScreenshotWithCompletion:(void(^)(UIImage *_Nullable screenshotImage))completion, will give you a picture of what is in the player, only including the logo on the top left corner.

Errors

You can find everything about Augment Player SDK errors in AugmentPlayerError

Feedback and Contact

We are always interested in your feedback and suggestions for how we can improve this documentation. Please email us at support@augment.com.

Changelog

Check the SDK changelog here