Normal map seam problem + artefacts in xnormal

[marnamai]

I have a problem with a rather ugly seam on my normal map, I am using 3ds max, mudbox and xnormal.

workflow
1 ) made the low poly in 3ds max (editable poly)
2 ) unwrapped it
3 ) exported as .obj from 3ds max
4 ) imported lowpoly .obj into mudbox
5 ) sculpted highpoly based on low poly
6 ) exported highpoly as .obj from mudbox
7 ) baked the normal in xnormal using the low and highpoly .obj's (has some artifacts, no idea how to fix it)
8 ) applied normal map in material in 3ds max

normal map seam (all channels normal)

normal map red and green flipped

wire on normal map

[jimonions]

Is your entire model on one smoothing group?

Also how exactly are you rendering this, Could be a problem with inverted normals.

[marnamai]

yes it is 1 smoothing group, I am using the default render settings in max and just press the F9 key

the material is just a grey diffuse and the normal map assigned in the bump slot as a "normal bump"

[jimonions]

I've never rendered anything with 3ds max, I've always used marmoset and/or UDK. If you want you can send me your model and ill render it out for you as a test.

[marnamai]

format? is just the lowpoly obj and the normal map okay?

[jimonions]

yup obj is fine, same one you used for your render. Id like to try your normal map and try making one on my own so if you could send me your high too that would be great.

[marnamai]

the highpoly is around 600 mb :/ will take quite a while to upload

edit oh wait, without the diffuse it's only 200mb

[jimonions]

If its taking too long I would suggest zipping it.

[marnamai]

I had to upload it elsewhere because dropbox is being so fucking slow

https://mega.co.nz/#!LJYiTLzC!nOaYHx_NH ... 1AYQcX2__k

[jimonions]

Ok its nothing to do with your normal map. The mesh itself is acting very weird. There appears to be a seam around your each of your parts (tail, ears, legs). just like the one in your images.

I noticed upon loading into max that your smoothing groups were from 1 to 31 but your mesh looked fine in the viewport. I don't know if that's just some weird bug. You exported with smoothing groups checked right?

Upon trying to remove the seams I found that they would keep reappearing no matter what. You said you made this in max right? What version? Im using 2013.

From what I conclude it looks like you made it in parts and didn't smooth, stitch them properly? I'm really confused since I've never had this problem before.
Heres how it looks on my screen.

http://imgur.com/a/4vEm2#0

[Surfa]

Some of your artefacts are because the xnormal's tangent calculations aren't in-sync with those of the 3ds max renderer. Because blender and xnormal use the same code to calculate the tangents these artefacts aren't visible within blender.

Example:


A work around is to apply a vertex split (different smoothing groups) at the uv boundaries. To fix these completely you could bake the normal in the 3ds max baker which is sync'd with the renderer, or use the program hand-plane to convert an object space normal map (will need to be baked) into a tangent space that is sync'd with the renderer.

There are some other artefacts that need fixing by either changing the ray distance of the baker or make sure your cage fits the low-poly properly. The artefact on the head for example.

Edit: You might consider adding some padding to the bake as well because I have noticed some seams that appear because of mip-mapping in my small test.

Edit 2: Just seen post above, it could also be that your export caused the problem but when I imported it into blender it was just set as one continuous mesh (one smoothing group).

[marnamai]

these are the export settings I used for the lowpoly obj, the model had only 2 smoothing groups, 1 for the body and 2 for the bottom of the hooves

On another forum it was suggested to use a cage in xnormal, which fixed the artifacts.

There is a 2 pixel padding, I 'll increase that for the next bake

I am gonna do some more experimentation with using a seperate smoothing group for each uv island.

@ jimonions: I made it in max 2014, using box modelling, the entire mesh should be one (no seperate element) and only have 2 different smoothing groups in the version I uploaded.

@ surfa: baking in max isn't really an option, since it performs very sluggish with a highpoly mesh of 5 million polygons :/

edit 10000: when I apply the normal map as a diffuse it also shows the seam in the viewport

[jimonions]

I don't think I can help here unfortunately, the problem is beyond my current knowledge.
The only surefire way I know that will solve it is rebuilding the mesh by hand with a retopo tool.
On a note, if you're planning on using this for source you should use the program surfa mentioned, Handplane; Not that it will solve your problem.

[marnamai]

*trying object space + handplane (since object space had no seams iirc from an earlier test)*


object space normal map rendered in mudbox has no seams



onto handplane!

*sigh*

[marnamai]

so each uv island is a different smoothing group => making hard edges between the islands

light grey = smoothing group 1
dark grey = smoothing group 2

is that how it's supposed to be done?



or should they all have a unique smoothing group?

EDIT: just tried with all unique smoothing groups, instead of just 1 and 2 => no difference