Adding 360 Video to Unity

Blog Posts ,Software ,Unity
October 4, 2016

As the line between 360 video and computer-generated interactive content blurs, one thing I find myself frequently being asked is how to add a 360 degree video into a Unity scene for playback. This quick tutorial will walk through a quick way to add 360, 2D video content (e.g. the types video shot from a Gear 360 or Ricoh Theta camera) to a scene in the Unity game engine.

Prepare the Video File

Unity supports video playback through the use of the Movie Texture class and allows you to import .mov, .mpg, .mpeg, .mp4, .avi, and .asf files via QuickTime. If you don’t have it installed already, you can grab QuickTime here on Windows to import your 360 video file into the Assets folder of your project. I used a sample video that I shot on my Gear 360 and uploaded to YouTube, which was stitched using the Gear 360 app on my S6 Edge phone and in an .mp4 format. You’ll also want to use a video editing tool to horizontally mirror your clip, as our shader will flip it and this is the simplest workaround. When you have your clip stitched, in a supported file format, and mirrored, import it into your Unity project by using Assets > Import New Asset.

mosbyvcamera

Creating our Playback Sphere

There are three parts to our video sphere that we’ll need to put together in order to support video playback. By default, Unity will assign our movie as a Movie Texture object, so we don’t need to do anything specific there. What we’ll be doing is:

  1. Creating a sphere object that surrounds our camera, which will be our playback surface
  2. Making a new material for our movie texture with a custom shader to change the playback to the interior of the sphere
  3. Writing a script for our sphere to play our movie texture

When you create the sphere, give it a large scale (I use 20, 20, 20) and place it so that the center point is our camera. If you run your application now, you’ll notice that Unity, by default, doesn’t render the interior of the sphere, which we’ll fix with our shader when we create our new material.

In the Assets folder, we’ll right-click and create three new files:

  1. Assets > Create > Shader > Standard Surface Shader, named “InvertNorm”
  2. Assets > Create > C# Script, named “PlayVideo.cs”
  3. Assets > Create > Material, named “VideoMat”

Double-click on the InvertNorm shader file to open it in your script editor (I use Visual Studio Community Edition, which is available for free here) and replace the contents with this code:

Shader "Flip Normals" {
	Properties{
		_MainTex("Base (RGB)", 2D) = "white" {}
	}
		SubShader{

		Tags{ "RenderType" = "Opaque" }

		Cull Front

		CGPROGRAM

#pragma surface surf Lambert vertex:vert
		sampler2D _MainTex;

	struct Input {
		float2 uv_MainTex;
		float4 color : COLOR;
	};


	void vert(inout appdata_full v)
	{
		v.normal.xyz = v.normal * -1;
	}

	void surf(Input IN, inout SurfaceOutput o) {
		fixed3 result = tex2D(_MainTex, IN.uv_MainTex);
		o.Albedo = result.rgb;
		o.Alpha = 1;
	}

	ENDCG

	}

		Fallback "Diffuse"
}

With 3D models, the default behavior of a material will be to have it render on the outside of the mesh object that it’s placed on, defined by the object’s normals – the perpendicular vector from the plane of the geometry. What we’re doing with the above script is telling the object that we want to invert those normals, so that instead of facing outwards, the texture should face inwards of our geometry instead.

Save your shader file, then double click on PlayVideo.cs to open the script. Usually with a video texture, the video will auto-play, but when we add our custom shader to the material, it will override this and we’ll need to trigger our video to play automatically. Add the following two lines to your PlayVideo.cs script so that it looks like this:

using UnityEngine;
using System.Collections;

public class PlayVideoOnLoad : MonoBehaviour {
    public MovieTexture _movie;
	// Use this for initialization
	void Start () {
        _movie.Play();
	}
	
	// Update is called once per frame
	void Update () {
	
	}
}

This will automatically play our video, but you could call _movie.Play() whenever you’d like, such as when a particular object was interacted with, in your application. Note that we also need to specify what Movie Texture we’ll be playing back in the inspector.

Save your script file, and switch back into Unity. In the Asset pane, select the VideoMat and change the following settings in the Inspector:

  1. Change the Shader from Standard (Specular) in the top drop down to InvertNorm.
  2. Drag the video file into the Base (RGB) texture spot

Drag your video material onto your sphere. You should see it showing up on the inside as you’d expect it to with a 360 video view! Add your PlayVideo script to the sphere by dragging the script file onto it and play the video in Unity. Your video playback should start automatically and surround you in the scene!

The ability to look around and interact with things in your scene will depend on your main camera object, so I went ahead and used a slightly modified FPS Character controller for the demo, but you can reproduce this with any camera of your choosing.

Mo2

Resources

The full project example in this post can be found on GitHub here: https://github.com/misslivirose/360-video-example 

Shader Source: Answers.Unity3d.com

Original Video File: https://www.youtube.com/watch?v=cazYQ7v49zw 

Related Posts

5 thoughts on “Adding 360 Video to Unity”

  1. Craig Oda says:

    Thanks for writing your article. I had some problems getting the videos from my RICOH THETA S to start playing, so I am using an alternate script. If anyone has the same issue, I documented my experience with RICOH THETA video files.
    http://theta360.guide/blog/video,/unity/2016/10/16/video-file-on-unity-sphere.html

    Basically, I’m just looping the MovieTexture video360 with video360.loop = true;

  2. Cy Brg says:

    really sweet, thanks a lot for posting

  3. Jason Cochran says:

    is a samsung galaxy phone required to get this to work with the samsung 360 camera?

    1. misslivirose says:

      You’ll need to have some way to stitch the footage (either with the included PC software or a phone that can run the companion app) but it’s not strictly necessary.

  4. Will says:

    This is an awesome tutorial thank you very much for posting.

    For me it was mirroring my 360 video so all text appeared backwards.

    To fix it I had to set the scale of X on the sphere to -20 instead of 20.

Leave a Reply