Stereoscopic domemaster images

[Roberto Ziche]

Hi everybody,

 

I'm posting here some tests renderings using a domemaster lens shader for mental ray that allows for 3D steroscopic images.

It's still a work in progress, and I'd like to make this an open effort, posting all the info, research, user interface, and the source code (later on).

 

A quick intro about myself. I've been working on computer graphics for too long now, and while I used to be a developer (15 years ago), I'm now mainly doing 3D content. I started evaluating the possibility of steroscopic domemaster images after I was asked by Terry Galloway at the Chabot Space and Science Center in CA to do some research.

 

I started with Daniel F. Ott Angular fisheye shader, and expanded (a lot) from there.
Daniel's shader can be found here, and it is also included in the DLL I will post soon.

 

This shader starts from the camera position, and creates two virual cameras (left and right, renderable one at a time) that rotate, constantly looking at the specific point of the dome matching the current rendered pixel.
The shader supports horizontal and vertical domes, and any degree of tilt in-between.

 

Here are two samples of Center, Left, and Right images, in vertical and horizontal mode.

It's not easy to see the differences here, but look below at the test stereo pairs.

 

The Center image would match Daniel's shader, but it has a different orientation, as I use some tricks to rotate the coordinate system 90 degrees. The current orientation matches the camera viewport, so it's extremely intuitive to setup the camera.

 

Of course, withouth some restrictions, this system will creates distortion points and areas where the 3D effect is wrong, reversed, or misaligned. The shader allows the use of maps to control the cameras separation (reduce or eliminate 3D effect), head rotation (force some areas to be looked with the head looking straight), and head tilt.

 

These are sample maps that I'm using for testing.

 

Using the Turn and Separation maps above, you can see what happens when rendering a simple grid (Red=Right, Green=Left).

 

The image above is something I would consider for horizontal or slightly tilted domes, where above/behind the viewer head the 3D effect is eliminated to allow the top of the dome to be looked at without turning the head, and the back with fully turned head.

 

But maps can be used creatively to control any area of the dome. Here, the Turn map above is used in combination with a simple gradient as a Separation map to have 3D only in the front part of the dome.

 

I think the math is almost there. The User Interface might need some adjustments, and maybe some automatic correction of distorted areas can be used instead of the maps, but the major issue now is to find a way to create proper maps.

 

Unfortunatley, I don't have any dome at home to try it, so I rely on simulated previews using a 3D concoction in 3ds Max, but that won't let me verify if the head tilt and rotation is set correctly. I think those values can be only found by experimentation on real domes.

 

These are some samples from the horizontal and verticals samples above. Relax (don't cross) your eyes to see the 3D effect. It helps is you are a bit shortsighted and take you glasses off :)

 

Horizontal:

 

 

 

Vertical:

 

 

 

I will post more tomorrow. If you have any comments, please post.

Rob

[Roberto Ziche]

Some more info.

This is the current UI for the shader:

Field of View: Self explanatory

Camera: Choices are Center/Left/Right. Selects the camera to use for rendering. Center skips 90% of the calculations and gives you a highly optimized standard Angular Fish Eye shader.

Dome Radius: This is actually the distance at which the cameras line of sights converge, so it's not really the dome size. Maybe I'll rename it.

Dome Forward Tilt: Dome tilt in degrees. Note that this value is not used unless you enable Dome Tilt Compensation.

Cameras Separation: The initial separation of the L/R cameras.

Separation Multiplier: A value between 0-1 that multiples the Camera Separation. of course, it's meant to be used with a grayscale texture mapped to the screen space using the right button. It's used to control teh amount of 3D effect, and eliminate it where desired.

Turn Multiplier: A value 0-1 that controls the amount tof the head turn. To be used with a grayscale texture. Typical use, keep the head straight while looking at the top of the dome.

Head Tilt: A value 0-1 (with 0.5 being the "neutral" value) that tilts the cameras (or head) left/right. 0 means 90 degrees to the left, 1 means 90 degrees to the right (if I remember correctly).

Dome Tilt Compensation: Enabling this option, shifts all the calculations by the # of degrees specified in Dome Forward Tilt. basically, it keeps the cameras/head vertical while the dome rotates forward.
Maps used for the various multipliers and tilt settings will have to be custom made for the proper dome tilt.

Vertical Mode: Enable the vertical dome mode, which automatically adjust the head turn and adds a turn compensation for the upper and lower part of the dome.
It's a simplified and optimized version of the Dome Tilt Compensation with a 90 degrees tilt. Faster and easier to use.

These last two options might need some redesign.


If you use Daniel's original shader, you notice the absence of the Flip and Offset options. They are there, just hidden as the code does not implement them yet.

The Flip implementation should be a no brainer, but the Offset is something I have to think about, as the 3D effect could be affected.

Rob

[Roberto Ziche]

I'm posting the first compiled version of the beta shader in the attached zip file.
This is the 32-bit version for 3ds Max 2009.

To install:
Unzip it somewhere
move the DLL to:
\mentalray\shaders_autoload\shaders
move the MI file to:
\mentalray\shaders_autoload\include

When you start Max, you will have 3 new Lens shaders:
"Domemaster Stereo Shader"
"domeAFL_FOV"
"domeAFL_WxH"

The last two are Daniel's original fish-eye shaders, and if you already had them, you might want to remove those DLLs to avoid conflicts. The version in this new DLL is the exact same shader (with a small change to use atan2 instead of a slower if-else).

As usual, to see the shader UI described previously, drag and drop the shader from the render Setup dialog to a slot in the material editor.

When you run a render, if you enable information messages in the mental ray message window, you will see the shader version:
PHEN 0.3 info : domeAFL_FOV_Stereo ver: 0.1.0

Rob

[Roberto Ziche]

A note about using the multiplier and tilt maps.

The maps are standard black and white images. You can use whatever format 3ds Max supports. Keep them square (even if it's not a requirement), and remember to map them as Environmnet->Screen in the Coordinates panel or you'll spend some time looking at images that makes no sense :)

The sample multiplier images in the first post above have a reference grid, but that was just for display. The map shodul not have anything other than the grayscale image. You can use a grid in a Photoshop layer as a reference, but then disable it before saving the actual map.

You can use 8-bit or 16-bit grayscale images (or grayscale RGB images). Unfortunatley I found that Photoshop with 16-bit images is unable to use some filters. You can use 8-bit if you need those filters, and 3ds Max should still be able to interpolate and give you smooth results.

Also, you don't need to make the grayscale images as big as the domemaster image. You can make them 800x800 or 1024x1024.

Rob

[Antoine Durr]

Cool!  Do you have a wireframe picture that is your baseline, i.e. no maps applied?  Also, the left eye is usually to the right of the right eye until you get to the convergence point, so your red/green may be swapped.

[Antoine Durr]

Dome radius seems correctly named.  The use of maps is certainly very flexible, but probably *too* flexible unless you just have a default set of maps that get used, i.e. it's doubtful that this is something that would be changed on a per-shot basis.  Figure it out for the show, and use that from then on.  The typical camera separation should be 2.5 inches, as that is the human interocular distance.  I presume that the separation multiplier would be used for feet-to-inch conversion, etc.?

Roberto Ziche said:

Some more info.

This is the current UI for the shader:

Field of View: Self explanatory

Camera: Choices are Center/Left/Right. Selects the camera to use for rendering. Center skips 90% of the calculations and gives you a highly optimized standard Angular Fish Eye shader.

Dome Radius: This is actually the distance at which the cameras line of sights converge, so it's not really the dome size. Maybe I'll rename it.

Dome Forward Tilt: Dome tilt in degrees. Note that this value is not used unless you enable Dome Tilt Compensation.

Cameras Separation: The initial separation of the L/R cameras.

Separation Multiplier: A value between 0-1 that multiples the Camera Separation. of course, it's meant to be used with a grayscale texture mapped to the screen space using the right button. It's used to control teh amount of 3D effect, and eliminate it where desired.

Turn Multiplier: A value 0-1 that controls the amount tof the head turn. To be used with a grayscale texture. Typical use, keep the head straight while looking at the top of the dome.

Head Tilt: A value 0-1 (with 0.5 being the "neutral" value) that tilts the cameras (or head) left/right. 0 means 90 degrees to the left, 1 means 90 degrees to the right (if I remember correctly).

Dome Tilt Compensation: Enabling this option, shifts all the calculations by the # of degrees specified in Dome Forward Tilt. basically, it keeps the cameras/head vertical while the dome rotates forward.
Maps used for the various multipliers and tilt settings will have to be custom made for the proper dome tilt.

Vertical Mode: Enable the vertical dome mode, which automatically adjust the head turn and adds a turn compensation for the upper and lower part of the dome.
It's a simplified and optimized version of the Dome Tilt Compensation with a 90 degrees tilt. Faster and easier to use.

These last two options might need some redesign.


If you use Daniel's original shader, you notice the absence of the Flip and Offset options. They are there, just hidden as the code does not implement them yet.

The Flip implementation should be a no brainer, but the Offset is something I have to think about, as the 3D effect could be affected.

Rob

[Roberto Ziche]

Hi Antoine,

 

If I apply no maps, the center of the image will be distorted, as the cameras will rotate while looking at the small center area. As a minimum, a simple map should be applied for the center, and can be a procedural map, like a gradient, that progressively zeroes the camera separation when approaching the center. That will eliminate 3D for that spot, but avoid a very visible artifacts. The image below uses this method.

 

The best approach remains, in my opinion, the combination of head turn multiplier and camera separation map I illustrated above, at least for horizontal or slightly tilted domes.

 

The camera separation multiplier is there just to control the amount of 3D effect, not a conversion factor. The base value for the camera separation can be the eye distance or a larger value then controlled by the map if the effect needs to be enhanced or diminished from "normal".
The distance hes to match the scene units, so if I'm working in millimeters, I would enter 65 there, but also the dome radius need to be adjusted.

A set of default maps can be provided, but a serious production should consider adjusting them on a shot by shot basis. As an extreme example, if the action starts on one area of the dome and crosses to another area, the maps could be animated to keep the best 3D possible effect following the action.
That means that head turn, tilt, and eye separation are constantly adjusted for the predicted viewer head movement.

On anaglyph images, red and blue images will swap accordingly to the position of the object relative to the convergence point (dome radius). That's teh same as it woudl work on any other steroscopy camera setup on a 3d application.

I'm currently doing some short animations and test images that I can test on a real dome. That will finally help me tweak this thing :)
Here's an anaglyph of a frame (reduced).

[Antoine Durr]

Regarding camera separati

Roberto Ziche said:

If I apply no maps, the center of the image will be distorted, as the cameras will rotate while looking at the small center area. As a minimum, a simple map should be applied for the center, and can be a procedural map, like a gradient, that progressively zeroes the camera separation when approaching the center. That will eliminate 3D for that spot, but avoid a very visible artifacts. The image below uses this method.

 

The best approach remains, in my opinion, the combination of head turn multiplier and camera separation map I illustrated above, at least for horizontal or slightly tilted domes.

 

The camera separation multiplier is there just to control the amount of 3D effect, not a conversion factor. The base value for the camera separation can be the eye distance or a larger value then controlled by the map if the effect needs to be enhanced or diminished from "normal".
The distance hes to match the scene units, so if I'm working in millimeters, I would enter 65 there, but also the dome radius need to be adjusted.

From a UI standpoint, you have two controls that impact the same result.  Why use one over the other?  That usually just confuses the users, especially if you can just put an expression into the field, e.g. 65*1.25.  

 

A set of default maps can be provided, but a serious production should consider adjusting them on a shot by shot basis. As an extreme example, if the action starts on one area of the dome and crosses to another area, the maps could be animated to keep the best 3D possible effect following the action.
That means that head turn, tilt, and eye separation are constantly adjusted for the predicted viewer head movement.

I don't think you can always predict where people will look.  They may find something interesting completely away from your center of action, or may still be looking at some interesting object even though now we're all supposed to look somewhere else.  And good planetarium shows have lots of that, which only adds to the immersive experience.

On anaglyph images, red and blue images will swap accordingly to the position of the object relative to the convergence point (dome radius). That's teh same as it woudl work on any other steroscopy camera setup on a 3d application.

I'm currently doing some short animations and test images that I can test on a real dome. That will finally help me tweak this thing :)
Here's an anaglyph of a frame (reduced).

 

[Roberto Ziche]

Hi Antoine,

 

The multiplier makes sense only when used witha texture map. Changing the value in the UI with no map is just for testing purpose.

 

Unfortunatley the UI is limited to what Max/mental ray allow me to do . I'm using a simple float value mappable to a texture. That's the easiest way to implement it as it's 100% transparent to the code, and mental ray handles all the stuff for me.

The drawback, is that the float value input field is visible in the UI, and I don't have a way to hide it.

 

If I implement it as a map, the UI would show just a big button for the map assignment, but my code would become a lot more complex.

 

About the second question. Viewers can look anywhere on the dome, and 90% of the time they can and they will see proper 3D. Only some areas need corrections, and the show producers shoudl be aware of those areas.

 

I had the opprtunity today to finally test some images and animations on a real dome, and everything was working as I expected.

I had areas with no 3D effect and they blended with no problem with the areas in full 3D. I'm confident that the system works.

 

I'm now going to be working of a few 30 seconds demo animations for different dome configurations.

[Antoine Durr]

Hey Roberto,

  I was wondering, what is happening with the stereo offset direction SE and SW of center?  Left vs. right follows the lattitude around most of the dome.  Is that the effect of the separation map or the turn map?

 

[Antoine Durr]

Have you done any tests with an extremely wide interocular?

[Roberto Ziche]

That's the separation map. It's spreading too far. The maps I posted in the first post are just two tests and need tuning.

I have to look for a better technique to create them, as photoshop doesn't let me control the gradient areas as I want.

Maybe I can use Illustrator, or even using a different projection (Cylindrical?) and then convert to polar.

 

I run a couple of test saturday using a 5in and 10in eyes separation. Despite being both hyperstereo, the 5in still looked not deep enough. I believe that's becasue I rendered for a dome witha a bigger diameter than the one I tested the images on.

 

Next time I will prepare some ad-hoc images for that dome.

 

[Michael J. Narlock]

Roberto,

 

Do you have a x64 version of this plugin that will work with max2011?  

 

I currently use two fisheye renderers...vraydomecam and Ott's domeafl (though for some reason I can't get his domeafl_WxH to work...comes up as "not found" shader).

 

At any rate, I'd love to test out your plugin on our systems.  Currently the fisheye renderers out there aren't "one stop shopping"...Mental Ray doesn't do afterburn and vray doesn't work well with vue xstream.