Compiling Shaders

[optio7]

Hey all,

new here at interlopers but looks like a good place to help my kind: the totally confused.

I've been trying to compile custom shaders for my mod, it uses the Orange Box engine and I heard it wasn't possible to compile customs before, but I reckoned the current release might work?

I've been following Wraiyth's & Valve developer's tutorial on their sites, but I keep ending up with the same error:

"...

Building inc files, asm vcs files, and VMPI worklist for stdshader_dx9_20b...
Publishing shader inc files to target...
Generating action list...
Running distributed shader compilation...
SteamStartup() failed: SteamStartup(0xf,0x0018DF58) failed with error 1: failed to take master pipe connection lock

shaders\psh\SDK_emissive_scroll_blended_pass_dx8_ps11.vcs
.... "

This is what happens during the batch compile, I get all the files it was supposed to create appart from the ' .inc ' ones

The .bat files for the compile havent been changed at all, apart from the directory of the mod.

Anybody had this problem before, or know a solution?

Any help as appreciated,
Cheers

[kraid]

AFAIK, custom shaders cannot be used under Orangebox Engine.
Valve simply didn't add this feature.

[-=Biohazard=-]

AFAIK, custom shaders cannot be used under Orangebox Engine.
Valve simply didn't add this feature.

Nope, custom shaders are working fine under obox.
When I get this startup error during model compiling, starting a game or whatever a steam restart always fixes it. Other than that it could be possible that the paths in your batch file aren't set up correctly or something, hard to tell.

[Surfa]

Are you running steam whilst you try this, if not then run steam at the same time. Also try running the source sdk if that doesn't work. Also yeah most errors with shaders come from path problems in the .bat file although this one doesn't look like it is double check your file paths.

[coder0xff]

As for the problems with file paths, the issue only arises when you have paths with spaces in them. Use the 8.3 formatted name as a work around. Use

Code: Select all

dir /x

to get the 8.3 name.

[ScarT]

Wraiyth's tutorial isn't updated to the Orange Box codebase so they're heavily outdated. Compiling shaders is possible on the Orange Box, except for one bug with the shadercompile.exe.

[coder0xff]

Necessary changes that I remember for the post process tutorial:
The fixes specified in the tutorial are no longer needed.
In buildsdkshaders.bat, change first command to:

Code: Select all

call "%VS80COMNTOOLS%vsvars32.bat"

If VS80COMNTOOLS does not appear when you type 'SET' into a command prompt, then either install VS2008 and copy the value from VS90COMNTOOLS into a new variable called VS80COMNTOOLS. These are environment variables. If you don't know how to set 'em up in windows XP/Vista/7, look it up.
still in buildsdkshaders.bat set GAMEDIR and SDKBINDIR as instructed
add a line to buildsdkshaders.bat such as:

Code: Select all

%BUILD_SHADER% modshaders -game %GAMEDIR% -source %SOURCEDIR% -dx9_30 -force30

Use SHADER_SAMPLER0 instead of SHADER_TEXTURE_STAGE0 in the .cpp

[optio7]

Hey, thanks everyone for replies

Yeah it turned out to be a problem with the paths I think, I messed around with peoples suggestions.

When I started compiling I got no errors, but it comes up saying somethin like

Compiling '1+ billion' sections 25 million combos.

It starts to compile really slowly and looks like it would take the entire day to do. Not sure if this is right?

Thanks again

[-=Biohazard=-]

There are two .txt files that list all shaders which you actually want to compile. Compiling everything would take quite a long time indeed, so I suggest you to create your own file which only contains your stuff, and use that one instead.

[coder0xff]

You kinda have to compile all the shaders if you don't want to have to remove them from the game. Adding a shader requires compiling the stdshader_dx9 project, which relies on include files generated by the shader compilation.

And yes, it takes a VERY long time. I don't what's going on, but who does?

[-=Biohazard=-]

Well you just need to create the includes to successfully build the project, so that's alright. I assume he got all includes, because the compiling itself bugged the first time, but the perl scripts still created the includes. (The output "Publishing shader inc files to target..." would mean, that the batch created the includes, I guess?).

Trying to use one of the default shaders though will result in an error about missing shaderfiles, if you didn't compile those. But that all depends on what you actually want to do in the end.

Just for messing around with some postprocessing stuff I didn't compile all of them myself in the beginning, but added them later, so that's no issue.

[optio7]

Hey,

thanks all for your help, looks like I've got it working properly now.

Cheers
Optio

[Mystfit]

Sorry to dig up this topic again but I'm having problems all over the place while trying to compile shaders.

Did the use of "modshaders.txt" suddenly fall out of practice? No matter what I do, buildsdkshaders never seems to pick up my post processing shaders and generate include files. I got around it by putting them in stdshader_dx9_20b.txt which seems to work, although I have no real idea if that's a bad thing to do.

I've been following the original tutorial by Wraiyth, but I'm unsure about which parts are still relevant to the current OB codebase. Currently I'm stuck on compiling the stdshader_dx9-2005 project since I keep getting these errors.

Code: Select all

1>d:\3d\machinima sandbox\src\materialsystem\stdshaders\post_sepia.cpp(31) : error C2664: 'IShaderShadow::EnableTexture' : cannot convert parameter 1 from 'TextureStage_t' to 'Sampler_t'
1>        Conversion to enumeration type requires an explicit cast (static_cast, C-style cast or function-style cast)
1>d:\3d\machinima sandbox\src\materialsystem\stdshaders\post_sepia.cpp(34) : error C2660: 'IShaderShadow::VertexShaderVertexFormat' : function does not take 5 arguments
1>d:\3d\machinima sandbox\src\materialsystem\stdshaders\post_sepia.cpp(45) : error C2664: 'void CBaseShader::BindTexture(Sampler_t,Sampler_t,int,int)' : cannot convert parameter 1 from 'TextureStage_t' to 'Sampler_t'
1>        Conversion to enumeration type requires an explicit cast (static_cast, C-style cast or function-style cast)

My cpp code for the shader is below.

Code: Select all

#include "BaseVSShader.h"
#include "post_sepia_ps20.inc"
#include "post_screenspaceeffect_vs20.inc"

BEGIN_VS_SHADER( Post_Sepia, "Help for Post_Sepia" )

BEGIN_SHADER_PARAMS
   SHADER_PARAM( FBTEXTURE, SHADER_PARAM_TYPE_TEXTURE, "_rt_FullFrameFB", "" )
END_SHADER_PARAMS

SHADER_INIT_PARAMS(){ }

SHADER_FALLBACK
{
   return 0;
}

SHADER_INIT
{
   if( params[FBTEXTURE]->IsDefined() )
   {
      LoadTexture( FBTEXTURE );
   }
}

SHADER_DRAW
{
   SHADOW_STATE
   {
      pShaderShadow->EnableDepthWrites( false );
      pShaderShadow->EnableTexture( SHADER_TEXTURE_STAGE0, true );

      int fmt = VERTEX_POSITION;
      pShaderShadow->VertexShaderVertexFormat( fmt, 1, 0, 0, 0 );

      DECLARE_STATIC_PIXEL_SHADER( post_sepia_ps20 );
      SET_STATIC_PIXEL_SHADER( post_sepia_ps20 );

      DECLARE_STATIC_VERTEX_SHADER( post_screenspaceeffect_vs20 );
      SET_STATIC_VERTEX_SHADER( post_screenspaceeffect_vs20 );
   }

   DYNAMIC_STATE
   {
      BindTexture( SHADER_TEXTURE_STAGE0, FBTEXTURE, -1 );

      DECLARE_DYNAMIC_PIXEL_SHADER( post_sepia_ps20 );
      SET_DYNAMIC_PIXEL_SHADER( post_sepia_ps20 );

      DECLARE_DYNAMIC_VERTEX_SHADER( post_screenspaceeffect_vs20 );
      SET_DYNAMIC_VERTEX_SHADER( post_screenspaceeffect_vs20 );
   }

   Draw();
}

END_SHADER



Any idea why this isn't working? The tutorial on Moddb isn't very helpful since the text is broken. I've had to reconstruct as much of the code as possible from the breakdowns he gives for every line.

[-=Biohazard=-]

I got around it by putting them in stdshader_dx9_20b.txt which seems to work, although I have no real idea if that's a bad thing to do.

You have to do so, actually (modshaders.txt was probably the old name).

Replace the lines in question with those:

pShaderShadow->EnableTexture( SHADER_SAMPLER0, true );
[...]
pShaderShadow->VertexShaderVertexFormat( fmt, 1, 0, 0 );
[...]
BindTexture( SHADER_SAMPLER0, FBTEXTURE, -1 );

[Mystfit]

Awesome, thanks BioHazard!

I've now got the shader project compiling, but when I try to use the shader ingame, it draws only as a wireframe and says in the console that the shader "post_sepia" could not be found.

I thought to get the shader working, all I need to do was have the inc file generated by the compiler. I checked in my shader directory in sourcemods/mymod and all of the files seem to be from valve and not from me.

When I compiled the shader project, did include my shader in the dll? Or is the shader compiler not creating/moving files properly?