Shadow Mapping in Source

[Gary]

In attempt to organize, here are important/interesting topics inside this thread:

• MrTwoVideoCards explaining difficulties during the Source2007 to Source2010(AlienSwarm) engine porting process. Then announcing a project that aims to bring Deferred Rendering to Source2010. -May 01, 2012
  
• My implementation of CSM in the Source engine. Which I am no longer bothering with since the above announcement. -Apr 30, 2012
  
• Using env_global_light(not CSM, it's a single ortho projected texture): Dabu(Portal2 - Oct 15, 2011) - Me(AlienSwarm - Feb 12, 2012)
  
• Varsity makes a "Level of Detail" system for env_projectedtextures. Allowing lots of low resolution ones. -Aug 10, 2011

Old post(this thread was orignally just about env_projectedtexture):

Some of these fixes and others can be found at the VDC!


"pt" = env_projectedtexture
"VDC Page" = Either this page or this one.

-----------------------------------------------------------------------------------------
Adjustable shadow grain/filter
--Value found on the VDC page. -Publicized by Gary with guidance from Team Gear.
Can be definable per-env_projectedtexture, this has been by Team Gear and then myself(Gary).
-----------------------------------------------------------------------------------------

-----------------------------------------------------------------------------------------
Higher-then-framebuffer shadow map resolution

--Instructions found on VDC page. -Fix created and publicized by Saul.
Sadly, due to the way the system works, it is not easy to make this definable per-pt, so it effects all shadow mapped lights(Flashlight and pts).
-----------------------------------------------------------------------------------------

-----------------------------------------------------------------------------------------
Configurable texture keyvalue in Hammer
--Instructions found on VDC page. -Fix created and publicized by Saul. (Correct?)
This is not a huge advancement as you could always feed pts texture names through their input(not-in-FGD). But, this is still useful if you use non-default textures with pts often.
-----------------------------------------------------------------------------------------

-----------------------------------------------------------------------------------------
Decreasing shadow "bleeding" and "ghosting"
--Value found on the VDC page. -Publicized by Bites.
-----------------------------------------------------------------------------------------

-----------------------------------------------------------------------------------------
Shadow brightness/"atten"



--Not yet added to VDC page. -Myself(Gary)
As you can tell by the above images(click to enlarge), this can be rather useful for dynamic scenes, such as a this helicopter. Other uses would be for say, a hanging light that moves around.
-----------------------------------------------------------------------------------------

-----------------------------------------------------------------------------------------
Downloads:
Latest projectedtexture "mod" base I made[4.82 MB] -Should include all above fixes.

[All above fixes' source code[will be added very soon]]

-----------------------------------------------------------------------------------------



\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/

NOTE:
Everything below this point is very old/outdated and in many cases using env_projectedtextures incorrectly. "pt"s(env_projectedtextures) should never be used like they are in many of the images below, pts should not be used as a sun, they work best with lower FOVs and close proximity to shadow casters. A lot of the images below do not even contain fixes from the latest revision of the VDC article.
Most(if not all) were done for testing purposes and have no practical uses. Of course Env_projectedtextures can be extremely useful and in some cases needed for a scene to look decent(if most of a scene is models and needs shadows like in Portal2). ~7/25/2011

\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/

----

Grain/Jitter fix: Lower the value for "m_flShadowFilterSize" from it's current value down to something like 1.0f
Located in "imaterialsystem.h", I recommend trying different values, a table of recommend values is at the VDC though.

Results:

Before(No Mods):
http://img689.imageshack.us/img689/1748/test0000.jpg

After Grain/Jitter fix:
http://img217.imageshack.us/img217/1748/test0000.jpg

-----

The shadow map resolution(You must apply the depthbuffer fix from the VDC or your shadow resolution can't go above your screen resolution):

Near line 1380, comment out these two lines

m_nDepthTextureResolution = depthTex->GetActualWidth();
r_flashlightdepthres.SetValue( m_nDepthTextureResolution );

Now add this below it

m_nDepthTextureResolution = r_flashlightdepthres.GetFloat();

And remember to define the default values near line 96.

(I got this info from my friend 1/4 life, don't credit me!)

-----

Useful commands:

To prevent clipping:
r_flashlightscissor 0

These help with shadow bleeding and "ghosting":
mat_depthbias_shadowmap 0.00001
mat_slopescaledepthbias_shadowmap 3-4


Pictures:
-------------------------------

The first HL2 level redone with new dynamic lighting:
http://a.imageshack.us/img691/3894/sdkd ... 01with.jpg
http://a.imageshack.us/img837/3894/sdkd ... 01with.jpg
http://a.imageshack.us/img294/6797/sdkd ... 1withc.jpg
http://a.imageshack.us/img340/3308/sdkd ... 1withw.jpg
http://a.imageshack.us/img716/3894/sdkd ... 01with.jpg
http://a.imageshack.us/img409/3894/sdkd ... 01with.jpg

An Omni-light test, download posted below:
http://img836.imageshack.us/img836/6663/omnisml0000.jpg
http://img820.imageshack.us/img820/4144/omnisml0001.jpg
http://img829.imageshack.us/img829/1699/omnisml0002.jpg

Random tests(some of these include the depthbuffer fix):
http://img801.imageshack.us/img801/3745 ... ht0003.jpg
http://img515.imageshack.us/img515/4629 ... nt0000.jpg
http://img146.imageshack.us/img146/4452 ... nt0001.jpg
http://img440.imageshack.us/img440/9277 ... nt0002.jpg
http://img266.imageshack.us/img266/8279 ... nt0003.jpg
http://img210.imageshack.us/img210/237/basement0004.jpg
http://img225.imageshack.us/img225/9431 ... 030015.jpg
http://img820.imageshack.us/img820/6219 ... 030001.jpg
http://img534.imageshack.us/img534/9453 ... 030002.jpg
http://img716.imageshack.us/img716/8852 ... 030008.jpg
http://img695.imageshack.us/img695/898/ ... 030013.jpg
http://img229.imageshack.us/img229/5306 ... 030009.jpg
http://img840.imageshack.us/img840/8163 ... 030012.jpg
http://img294.imageshack.us/img294/2632 ... 030016.jpg
http://img823.imageshack.us/img823/5273 ... 030011.jpg

Attempt at using it for a sun, shadow res at 4096.
http://img830.imageshack.us/img830/3445 ... 030026.jpg
http://img263.imageshack.us/img263/6176 ... 030025.jpg
http://img146.imageshack.us/img146/3755 ... 030023.jpg
http://img201.imageshack.us/img201/1520 ... 030022.jpg
http://img253.imageshack.us/img253/8020 ... 030020.jpg




-------------------------------

Downloads:

I am planning on releasing a test-mod again, that will have all the fixes applied and the modified CPP files. It will contain omni-light tests, shadow resolution tests, practical uses and more. ~As of 10/29/2010

Sun Oct 24, 2010 6:33 pm Note: A more optimized version of this has been done, but not released.
Someone asked me to post the omnidirectional light I posted in this thread a few pages back. I re-did it because I didn't want to give something out that was a buggy pile of crap, so here it is, my version of an omnidirectional shadow mapped light.

Basically this is all the VDC fixes(including mine) and six 120-degree env_projectedtextures facing in different directions. Due the lights being 120 degrees, the light's view is stretched on the shadow map, resulting in even lower resolution shadows(restricted to even smaller scenes).

This demo has no gravity for props, found it was better for testing purposes.
http://img836.imageshack.us/img836/6663/omnisml0000.jpg
Click on the computer monitor that is sitting on the desk and everything explodes(not killing the player).
http://img820.imageshack.us/img820/4144/omnisml0001.jpg
http://img829.imageshack.us/img829/1699/omnisml0002.jpg


Included in download:
Custom Bins
The living room map(in pictures above)
The living room VMF


[Download | 3.56 MB], place in SourceMods folder.

I have been messing around with this entity and it is quite awesome, though it needs a lot of fixes. Most can be fixed, but I have yet to find a fix to the "grain".

Not the best example, I'll post another one later.


You can see it on the barrel's shadow, this effect become much greater when the distance is farther.

Does any one have any idea on how to fix this? I would greatly appreciate any help.

[Surfa]

Try a higher res texture?

[Gary]

It can't be that easy.

[city14]

I hope it's that easy... I've never used projected textured...

[Gradius]

It isn't. If I remember correctly it is the result of Nvidia's bitching about the way shadows were rendered.
It's called shadow jittering and it's how soft shadows are rendered, so unless you want razor ship shadows or to recode the entire rendering system for soft shadow endges I'd move on.

[Gary]

Oh, I see, though right now I would want razor sharp shadows over horrid ones.

Edit:

Didn't the City 17 mod fix the shadow jittering?

[Gary]

Well, I have been trying to force a higher resolution for the shadow maps, still no luck, it just doesn't seem to want to change.

And looking at Crysis, it is quite easy to see the same "grain" effect that plagues Source when you lower the shadow map resolution.

Funny that the "grain" effect is a attempt to smooth choppy shadows. It helps at close range, but not objects farther from the light source.

Images from the CryEngine:

e_shadows_max_texture_size 128
r_ShadowJittering 1
http://img138.imageshack.us/img138/516/128jitter1.png

e_shadows_max_texture_size 128
r_ShadowJittering 0
http://img194.imageshack.us/img194/158/128jitter0.png


e_shadows_max_texture_size 1024
r_ShadowJittering 1
http://img190.imageshack.us/img190/848/1024jitter1.png

e_shadows_max_texture_size 1024
r_ShadowJittering 0
http://img251.imageshack.us/img251/8514/1024jitter0.png


e_shadows_max_texture_size 2048
r_ShadowJittering 1
http://img97.imageshack.us/img97/5413/2048jitter1.png

e_shadows_max_texture_size 2048
r_ShadowJittering 0
http://img695.imageshack.us/img695/1576/2048jitter0.png



e_shadows_max_texture_size 4096
r_ShadowJittering 1
http://img823.imageshack.us/img823/5927/4096jitter1.png

At this point, you don't even need Shadow Jittering.

e_shadows_max_texture_size 4096
r_ShadowJittering 0
http://img692.imageshack.us/img692/1597/4096jitter0.png


I am pretty sure Source has them set at 1024, which means they must use a different method of shadow mapping then Crysis. So I am thinking I could set the Source ones at 2048 and if that doesn't work, I could try 4096, but I am afraid that 4096 will use to much frame time like it does it in Crysis.

[MrTwoVideoCards]

The shadow filter code is what you might want to look at. For City17 we've set a definable filter per-projectedtexture. I think the current default is 2.0 or something, we've set it to 1.0.

The great benefit with a smaller grain is that we get perf back!

[Gary]

Great thanks!

I'll look into that and see if I can get it to work for me.

Thanks again.

[Gary]

Well, I could never find the filter code you spoke of. Till now, when looking through the code in ASW. I wasn't even looking for it, but when I saw the world "filter", I knew I found something. I changed it to 1.0 like you guys did.

Here is the results:



As you can see, it looks awesome, the only thing that could make it better is a higher shadowmap resolution.

Though, the filter edit does work in HL2 2007 codebase, for some reason the shadowmap resolution is stuck at 512 even though it's defined in code as 1024. It looks like crap, not nearly as good as it does in ASW. Any idea on how to fix this? And if possible, maybe set it to something even higher, such as 2048?

[Armageddon]

So how exactly did you change this in the 2009 engine?

[Gary]

m_flShadowFilterSize

[Mess]

what's mine is yours
what's yours is mine
the more we share
the more the sun will shine

[Gary]

I've never heard that saying before.

Well, anyways, Arma, if you can't find "m_flShadowFilterSize" through searching, it's in "imaterialsystem.h".


So, Mess, would you happen to know how to over-ride the shadowmap resolution?

[Jike]

Just entirely theoretically. Couldnt one apply something like a LOD setting to these textures? So, it looks good when far away, even though the quality sucks?
Would that be possible?