diff --git a/README.md b/README.md index f810e88..ed6b576 100644 --- a/README.md +++ b/README.md @@ -3,424 +3,115 @@ WebGL Deferred Shading **University of Pennsylvania, CIS 565: GPU Programming and Architecture, Project 6** -* (TODO) YOUR NAME HERE -* Tested on: (TODO) **Google Chrome 222.2** on - Windows 22, i7-2222 @ 2.22GHz 22GB, GTX 222 222MB (Moore 2222 Lab) +Terry Sun; Arch Linux, Intel i7-4600, integrated graphics -### Live Online +A tile-based deferred shader. -[![](img/thumb.png)](http://TODO.github.io/Project6-WebGL-Deferred-Shading) +Check it out: [live online](http://terrysun.blue/WebGL-Deferred-Shader/)! +There are a lot of available sliders and options for +you to play with. -### Demo Video +![](img/normal.png) -[![](img/video.png)](TODO) +Showing 300 lights at a tile size of 75x75, rendering in about 80ms. -### (TODO: Your README) +## Tile -*DO NOT* leave the README to the last minute! It is a crucial part of the -project, and we will not be able to grade you without a good README. +This tile-based shader technique "inverts" the deferred shader step: rather than +drawing to the entire scene once per each light, the scene is drawn in +individual tiles, and each tile loops over the lights that intersect it. -This assignment has a considerable amount of performance analysis compared -to implementation work. Complete the implementation early to leave time! +![](img/tiled_textures.png) +(Figure from [Tiled Shading](http://www.cse.chalmers.se/~uffe/tiled_shading_preprint.pdf)) -Instructions (delete me) -======================== +Inspiration for texture layout was taken from the image above, which a few +modifications: the global light list was stored as two separate textures with +corresponding indices (one containing position and radius; the other containing +light colors). The "tile light light indices" texture, which stored the list of +lights for each tile, packed closely, was not an Nx1 texture but a 4092xM +texture, expanding to allow larger lists of indices. (The light grid, however, +never exceeded an Nx1 size.) -This is due at midnight on the evening of Tuesday, October 27. +This technique will render 300 lights in ~66 ms and 400 lights in ~76 ms. (See +below for more information.) -**Summary:** In this project, you'll be introduced to the basics of deferred -shading and WebGL. You'll use GLSL and WebGL to implement a deferred shading -pipeline and various lighting and visual effects. +## Toon Shading -**Recommendations:** -Take screenshots as you go. Use them to document your progress in your README! +In addition to Blinn-Phong lighting, this shader implements a non-photorealistic +toon shader. This consists of two main effects: -Read (or at least skim) the full README before you begin, so that you know what -to expect and what to prepare for. +* Discretization of color values, which mimicks a hand-drawn or hand-colored + shading style. This is a simple post-processing step (though not a separate + deferred pipeline step) where the calculated Blinn-Phong light value is + floored to the nearest color "edge". +* Edge shading applies a bolding effect on the edges of geometry, calculated by + sampling the depth texture at a fragment's neighbors. If the maximum depth + *difference* is greater than some threshold, then consider the fragment to lie + on an edge and shade it black. The edge can be made thicker by sampling pixels + which lie farther away. -### Running the code +![](img/toon.png) -If you have Python, you should be able to run `server.py` to start a server. -Then, open [`http://localhost:10565/`](http://localhost:10565/) in your browser. +### Watercolor -This project requires a WebGL-capable web browser with support for -`WEBGL_draw_buffers`. You can check for support on -[WebGL Report](http://webglreport.com/). +Actually a cool blooper: this is what happens if you discretize over the *sum* +of colors in a tile rather than each individual light. -Google Chrome seems to work best on all platforms. If you have problems running -the starter code, use Chrome or Chromium, and make sure you have updated your -browser and video drivers. +(Currently, supported on tile-based deferred shading only. On non-tile-based +shading, this would have to be a post-processing step.) -In Moore 100C, both Chrome and Firefox work. -See below for notes on profiling/debugging tools. +![](img/watercolor.png) -Use the screenshot button to save a screenshot. +## Performance -## Requirements +Performance numbers were taken with stats.js, with the camera placed inside the +scene and lights evenly distributed within the scene such that almost all of +them are within view of the camera. -**Ask on the mailing list for any clarifications.** +![](img/chart_opts.png) -In this project, you are given code for: +This results are similar to -* Loading OBJ files and color/normal map textures -* Camera control -* Partial implementation of deferred shading including many helper functions +![](img/chart_tile_size.png) -### Required Tasks +The tile size actually doesn't have a particularly large effect at lower light +counts. Here, the performance gain of partitioning fewer lights per tile (and +decreasing the amount of time looping through lights on the GPU) is counteracted +by the amount of work needed to compute the lights-per-tile partition texture on +the CPU, leaving the time relatively consistent for many tile sizes between +40x40 and 100x100. The timing at about 30x30 is particularly inconsistent -- +perhaps this is one place where the CPU/GPU tradeoff is close to even. At tile +sizes smaller than 20x20, you might expect there to be few lights per tile, it +takes quite a lot of time in order to generate the necessary textures (which are +themselves much larger). -**Before doing performance analysis,** you must disable debug mode by changing -`debugMode` to `false` in `framework.js`. Keep it enabled when developing - it -helps find WebGL errors *much* more easily. +With larger numbers of lights in the scene, larger tile sizes also see a large +hit to performance due to looping over lights in the fragment shader. With +fewer lights, this takes much longer (not shown) to manifest as a problem. -You will need to perform the following tasks: +At higher light counts, the majority of the time is spent on the CPU computing +light textures. This is verified by using the Chome Profiling tool, which shows +that a significantly higher percentage of time is spent in the texture +computation function (~75% at 300 lights, ~50% at 30 lights) at more brightly +lit scenes. With fewer lights in the scene, a much smaller percentage of time is +spent on the CPU. I suspect that that the actual ms-per-frame spent on the GPU +rises at a much slower rate per light than on the CPU, accounting for the large +difference in relative time on each procesing unit. -* Complete the deferred shading pipeline so that the Blinn-Phong and Post1 - shaders recieve the correct input. Go through the Starter Code Tour **before - continuing!** +![](img/chart_toon.png) -**Effects:** +Comparison take with 50x50-size tiles. -* Implement deferred Blinn-Phong shading (diffuse + specular) - * With normal mapping (code provided) +The main performance cost of toon shading is the additional texture accesses +required to find edges, as each pixel must read four neighbors' depth values. +The discretization of light values should add almost no cost, since it is +implemented as a simple mathematical funciton. -* Implement one of the following effects: - * Bloom using post-process blur (box or Gaussian) [1] - * Toon shading (with ramp shading + simple depth-edge detection for outlines) +## References -**Optimizations:** - -* Scissor test optimization: when accumulating shading from each point - light source, only render in a rectangle around the light. - * Show a debug view for this (showing scissor masks clearly), e.g. by - modifying and using `red.frag.glsl` with additive blending and alpha = 0.1. - * Code is provided to compute this rectangle for you, and there are - comments at the relevant place in `deferredRender.js` with more guidance. - -* Optimized g-buffer format - reduce the number and size of g-buffers: - * Ideas: - * Pack values together into vec4s - * Use 2-component normals - * Quantize values by using smaller texture types instead of gl.FLOAT - * Reduce number of properties passed via g-buffer, e.g. by: - * Applying the normal map in the `copy` shader pass instead of - copying both geometry normals and normal maps - * Reconstructing world space position using camera matrices and X/Y/depth - * For credit, you must show a good optimization effort and record the - performance of each version you test, in a simple table. - * It is expected that you won't need all 4 provided g-buffers for a basic - pipeline - make sure you disable the unused ones. - * See mainly: `copy.frag.glsl`, `deferred/*.glsl`, `deferredSetup.js` - -### Extra Tasks - -You must do at least **10 points** worth of extra features (effects or -optimizations/analysis). - -**Effects:** - -* (3pts) The effect you didn't choose above (bloom or toon shading) - -* (3pts) Screen-space motion blur (blur along velocity direction) [3] - -* (2pts) Allow variability in additional material properties - * Include other properties (e.g. specular coeff/exponent) in g-buffers - * Use this to render objects with different material properties - * These may be uniform across one model draw call, but you'll have to show - multiple models - -**Optimizations/Analysis:** - -* (2pts) Improved screen-space AABB for scissor test - (smaller/more accurate than provided - but beware of CPU/GPU tradeoffs) - -* (3pts) Two-pass **Gaussian** blur using separable convolution (using a second - postprocess render pass) to improve bloom or other 2D blur performance - -* (4-6pts) Light proxies - * (4pts) Instead of rendering a scissored full-screen quad for every light, - render some proxy geometry which covers the part of the screen affected by - the light (e.g. a sphere, for an attenuated point light). - * A model called `sphereModel` is provided which can be drawn in the same - way as the code in `drawScene`. (Must be drawn with a vertex shader which - scales it to the light radius and translates it to the light position.) - * (+2pts) To avoid lighting geometry far behind the light, render the proxy - geometry (e.g. sphere) using an inverted depth test - (`gl.depthFunc(gl.GREATER)`) with depth writing disabled (`gl.depthMask`). - This test will pass only for parts of the screen for which the backside of - the sphere appears behind parts of the scene. - * Note that the copy pass's depth buffer must be bound to the FBO during - this operation! - * Show a debug view for this (showing light proxies) - * Compare performance of this, naive, and scissoring. - -* (8pts) Tile-based deferred shading with detailed performance comparison - * On the CPU, check which lights overlap which tiles. Then, render each tile - just once for all lights (instead of once for each light), applying only - the overlapping lights. - * The method is described very well in - [Yuqin & Sijie's README](https://github.com/YuqinShao/Tile_Based_WebGL_DeferredShader/blob/master/README.md#algorithm-details). - * This feature requires allocating the global light list and tile light - index lists as shown at this link. These can be implemented as textures. - * Show a debug view for this (number of lights per tile) - -* (6pts) Deferred shading without multiple render targets - (i.e. without WEBGL_draw_buffers). - * Render the scene once for each target g-buffer, each time into a different - framebuffer object. - * Include a detailed performance analysis, comparing with/without - WEBGL_draw_buffers (like in the - [Mozilla blog article](https://hacks.mozilla.org/2014/01/webgl-deferred-shading/)). - -* (2-6pts) Compare performance to equivalently-lit forward-rendering: - * (2pts) With no forward-rendering optimizations - * (+2pts) Coarse, per-object back-to-front sorting of geometry for early-z - * (Of course) must render many objects to test - * (+2pts) Z-prepass for early-z - -This extra feature list is not comprehensive. If you have a particular idea -that you would like to implement, please **contact us first** (preferably on -the mailing list). - -**Where possible, all features should be switchable using the GUI panel in -`ui.js`.** - -### Performance & Analysis - -**Before doing performance analysis,** you must disable debug mode by changing -`debugMode` to `false` in `framework.js`. Keep it enabled when developing - it -helps find WebGL errors *much* more easily. - -Optimize your JavaScript and/or GLSL code. Web Tracing Framework -and Chrome/Firefox's profiling tools (see Resources section) will -be useful for this. For each change -that improves performance, show the before and after render times. - -For each new *effect* feature (required or extra), please -provide the following analysis: - -* Concise overview write-up of the feature. -* Performance change due to adding the feature. - * If applicable, how do parameters (such as number of lights, etc.) - affect performance? Show data with simple graphs. -* If you did something to accelerate the feature, what did you do and why? -* How might this feature be optimized beyond your current implementation? - -For each *performance* feature (required or extra), please provide: - -* Concise overview write-up of the feature. -* Detailed performance improvement analysis of adding the feature - * What is the best case scenario for your performance improvement? What is - the worst? Explain briefly. - * Are there tradeoffs to this performance feature? Explain briefly. - * How do parameters (such as number of lights, tile size, etc.) affect - performance? Show data with graphs. - * Show debug views when possible. - * If the debug view correlates with performance, explain how. - -Note: Be aware that stats.js may give 0 millisecond frame timings in Chrome on -occasion - if this happens, you can use the FPS counter. - -### Starter Code Tour - -You'll be working mainly in `deferredRender.js` using raw WebGL. Three.js is -included in the project for various reasons. You won't use it for much, but its -matrix/vector types may come in handy. - -It's highly recommended that you use the browser debugger to inspect variables -to get familiar with the code. At any point, you can also -`console.log(some_var);` to show it in the console and inspect it. - -The setup in `deferredSetup` is already done for you, for many of the features. -If you want to add uniforms (textures or values), you'll change them here. -Therefore, it is recommended that you review the comments to understand the -process, BEFORE starting work in `deferredRender`. - -In `deferredRender`, start at the **START HERE!** comment. -Work through the appropriate `TODO`s as you go - most of them are very -small. Test incrementally (after implementing each part, instead of testing -all at once). -* (The first thing you should be doing is implementing the fullscreen quad!) - -Your _next_ first goal should be to get the debug views working. -Add code in `debug.frag.glsl` to examine your g-buffers before trying to -render them. (Set the debugView in the UI to show them.) - -For editing JavaScript, you can use a simple editor with syntax highlighting -such as Sublime, Vim, Emacs, etc., or the editor built into Chrome. - -* `js/`: JavaScript files for this project. - * `main.js`: Handles initialization of other parts of the program. - * `framework.js`: Loads the scene, camera, etc., and calls your setup/render - functions. Hopefully, you won't need to change anything here. - * `deferredSetup.js`: Deferred shading pipeline setup code. - * `createAndBind(Depth/Color)TargetTexture`: Creates empty textures for - binding to frame buffer objects as render targets. - * `deferredRender.js`: Your deferred shading pipeline execution code. - * `renderFullScreenQuad`: Renders a full-screen quad with the given shader - program. - * `ui.js`: Defines the UI using - [dat.GUI](https://workshop.chromeexperiments.com/examples/gui/). - * The global variable `cfg` can be accessed anywhere in the code to read - configuration values. - * `utils.js`: Utilities for JavaScript and WebGL. - * `abort`: Aborts the program and shows an error. - * `loadTexture`: Loads a texture from a URL into WebGL. - * `loadShaderProgram`: Loads shaders from URLs into a WebGL shader program. - * `loadModel`: Loads a model into WebGL buffers. - * `readyModelForDraw`: Configures the WebGL state to draw a model. - * `drawReadyModel`: Draws a model which has been readied. - * `getScissorForLight`: Computes an approximate scissor rectangle for a - light in world space. -* `glsl/`: GLSL code for each part of the pipeline: - * `clear.*.glsl`: Clears each of the `NUM_GBUFFERS` g-buffers. - * `copy.*.glsl`: Performs standard rendering without any fragment shading, - storing all of the resulting values into the `NUM_GBUFFERS` g-buffers. - * `quad.vert.glsl`: Minimal vertex shader for rendering a single quad. - * `deferred.frag.glsl`: Deferred shading pass (for lighting calculations). - Reads from each of the `NUM_GBUFFERS` g-buffers. - * `post1.frag.glsl`: First post-processing pass. -* `lib/`: JavaScript libraries. -* `models/`: OBJ models for testing. Sponza is the default. -* `index.html`: Main HTML page. -* `server.bat` (Windows) or `server.py` (OS X/Linux): - Runs a web server at `localhost:10565`. - -### The Deferred Shading Pipeline - -See the comments in `deferredSetup.js`/`deferredRender.js` for low-level guidance. - -In order to enable and disable effects using the GUI, upload a vec4 uniform -where each component is an enable/disable flag. In JavaScript, the state of the -UI is accessible anywhere as `cfg.enableEffect0`, etc. - -**Pass 1:** Renders the scene geometry and its properties to the g-buffers. -* `copy.vert.glsl`, `copy.frag.glsl` -* The framebuffer object `pass_copy.fbo` must be bound during this pass. -* Renders into `pass_copy.depthTex` and `pass_copy.gbufs[i]`, which need to be - attached to the framebuffer. - -**Pass 2:** Performs lighting and shading into the color buffer. -* `quad.vert.glsl`, `deferred/blinnphong-pointlight.frag.glsl` -* Takes the g-buffers `pass_copy.gbufs`/`depthTex` as texture inputs to the - fragment shader, on uniforms `u_gbufs` and `u_depth`. -* `pass_deferred.fbo` must be bound. -* Renders into `pass_deferred.colorTex`. - -**Pass 3:** Performs post-processing. -* `quad.vert.glsl`, `post/one.frag.glsl` -* Takes `pass_BlinnPhong_PointLight.colorTex` as a texture input `u_color`. -* Renders directly to the screen if there are no additional passes. - -More passes may be added for additional effects (e.g. combining bloom with -motion blur) or optimizations (e.g. two-pass Gaussian blur for bloom) - -#### Debugging - -If there is a WebGL error, it will be displayed on the developer console and -the renderer will be aborted. To find out where the error came from, look at -the backtrace of the error (you may need to click the triangle to expand the -message). The line right below `wrapper @ webgl-debug.js` will point to the -WebGL call that failed. - -#### Changing the number of g-buffers - -Note that the g-buffers are just `vec4`s - you can put any values you want into -them. However, if you want to change the total number of g-buffers (add more -for additional effects or remove some for performance), you will need to make -changes in a number of places: - -* `deferredSetup.js`/`deferredRender.js`: search for `NUM_GBUFFERS` -* `copy.frag.glsl` -* `deferred.frag.glsl` -* `clear.frag.glsl` - - -## Resources - -* [1] Bloom: - [GPU Gems, Ch. 21](http://http.developer.nvidia.com/GPUGems/gpugems_ch21.html) -* [2] Screen-Space Ambient Occlusion: - [Floored Article](http://floored.com/blog/2013/ssao-screen-space-ambient-occlusion.html) -* [3] Post-Process Motion Blur: - [GPU Gems 3, Ch. 27](http://http.developer.nvidia.com/GPUGems3/gpugems3_ch27.html) - -**Also see:** The articles linked in the course schedule. - -### Profiling and debugging tools - -Built into Firefox: -* Canvas inspector -* Shader Editor -* JavaScript debugger and profiler - -Built into Chrome: -* JavaScript debugger and profiler - -Plug-ins: -* (Chrome/Firefox) [Web Tracing Framework](http://google.github.io/tracing-framework/) -* (Chrome) [Shader Editor](https://chrome.google.com/webstore/detail/shader-editor/ggeaidddejpbakgafapihjbgdlbbbpob) - - -Firefox can also be useful - it has a canvas inspector, WebGL profiling and a -shader editor built in. - - -## README - -Replace the contents of this README.md in a clear manner with the following: - -* A brief description of the project and the specific features you implemented. -* At least one screenshot of your project running. -* A 30+ second video of your project running showing all features. - [Open Broadcaster Software](http://obsproject.com) is recommended. - (Even though your demo can be seen online, using multiple render targets - means it won't run on many computers. A video will work everywhere.) -* A performance analysis (described below). - -### Performance Analysis - -See above. - -### GitHub Pages - -Since this assignment is in WebGL, you can make your project easily viewable by -taking advantage of GitHub's project pages feature. - -Once you are done with the assignment, create a new branch: - -`git branch gh-pages` - -Push the branch to GitHub: - -`git push origin gh-pages` - -Now, you can go to `.github.io/` to see your -renderer online from anywhere. Add this link to your README. - -## Submit - -1. Open a GitHub pull request so that we can see that you have finished. - The title should be "Submission: YOUR NAME". - * **ADDITIONALLY:** - In the body of the pull request, include a link to your repository. -2. Send an email to the TA (gmail: kainino1+cis565@) with: - * **Subject**: in the form of `[CIS565] Project N: PENNKEY`. - * Direct link to your pull request on GitHub. - * Estimate the amount of time you spent on the project. - * If there were any outstanding problems, briefly explain. - * **List the extra features you did.** - * Feedback on the project itself, if any. - -### Third-Party Code Policy - -* Use of any third-party code must be approved by asking on our mailing list. -* If it is approved, all students are welcome to use it. Generally, we approve - use of third-party code that is not a core part of the project. For example, - for the path tracer, we would approve using a third-party library for loading - models, but would not approve copying and pasting a CUDA function for doing - refraction. -* Third-party code **MUST** be credited in README.md. -* Using third-party code without its approval, including using another - student's code, is an academic integrity violation, and will, at minimum, - result in you receiving an F for the semester. +* [Toon Shading WikiBooks](https://en.wikibooks.org/wiki/GLSL_Programming/Unity/Toon_Shading) +* Tile-based Deferred Shading + * [Mozilla Deferred Shading Article](https://hacks.mozilla.org/2014/01/webgl-deferred-shading/) + * [Yuqin & Sijie's README](https://github.com/YuqinShao/Tile_Based_WebGL_DeferredShader/blob/master/README.md#algorithm-details). + * [Tiled Shading](http://www.cse.chalmers.se/~uffe/tiled_shading_preprint.pdf) diff --git a/glsl/copy.frag.glsl b/glsl/copy.frag.glsl index 0f5f8f7..2a688e9 100644 --- a/glsl/copy.frag.glsl +++ b/glsl/copy.frag.glsl @@ -10,6 +10,17 @@ varying vec3 v_position; varying vec3 v_normal; varying vec2 v_uv; +vec3 applyNormalMap(vec3 geomnor, vec3 normap) { + normap = normap * 2.0 - 1.0; + vec3 up = normalize(vec3(0.001, 1, 0.001)); + vec3 surftan = normalize(cross(geomnor, up)); + vec3 surfbinor = cross(geomnor, surftan); + return normap.y * surftan + normap.x * surfbinor + normap.z * geomnor; +} + void main() { - // TODO: copy values into gl_FragData[0], [1], etc. + gl_FragData[0] = vec4(v_position, 0); + gl_FragData[1] = vec4(v_normal, 0); + gl_FragData[2] = texture2D(u_colmap, v_uv); + gl_FragData[3] = texture2D(u_normap, v_uv); } diff --git a/glsl/deferred/ambient.frag.glsl b/glsl/deferred/ambient.frag.glsl index 1fd4647..c3c1c60 100644 --- a/glsl/deferred/ambient.frag.glsl +++ b/glsl/deferred/ambient.frag.glsl @@ -7,21 +7,20 @@ precision highp int; uniform sampler2D u_gbufs[NUM_GBUFFERS]; uniform sampler2D u_depth; +uniform float u_ambientTerm; varying vec2 v_uv; void main() { - vec4 gb0 = texture2D(u_gbufs[0], v_uv); - vec4 gb1 = texture2D(u_gbufs[1], v_uv); vec4 gb2 = texture2D(u_gbufs[2], v_uv); - vec4 gb3 = texture2D(u_gbufs[3], v_uv); float depth = texture2D(u_depth, v_uv).x; - // TODO: Extract needed properties from the g-buffers into local variables + + vec3 colmap = vec3(gb2); if (depth == 1.0) { gl_FragColor = vec4(0, 0, 0, 0); // set alpha to 0 return; } - gl_FragColor = vec4(0.1, 0.1, 0.1, 1); // TODO: replace this + gl_FragColor = vec4(u_ambientTerm * colmap, 1); } diff --git a/glsl/deferred/blinnphong-pointlight.frag.glsl b/glsl/deferred/blinnphong-pointlight.frag.glsl index b24a54a..4741e2f 100644 --- a/glsl/deferred/blinnphong-pointlight.frag.glsl +++ b/glsl/deferred/blinnphong-pointlight.frag.glsl @@ -4,6 +4,9 @@ precision highp int; #define NUM_GBUFFERS 4 +uniform int u_toon; + +uniform vec3 u_cameraPos; uniform vec3 u_lightCol; uniform vec3 u_lightPos; uniform float u_lightRad; @@ -12,6 +15,8 @@ uniform sampler2D u_depth; varying vec2 v_uv; +const float TOON_STEPS = 3.0; + vec3 applyNormalMap(vec3 geomnor, vec3 normap) { normap = normap * 2.0 - 1.0; vec3 up = normalize(vec3(0.001, 1, 0.001)); @@ -20,20 +25,77 @@ vec3 applyNormalMap(vec3 geomnor, vec3 normap) { return normap.y * surftan + normap.x * surfbinor + normap.z * geomnor; } +// vec3(diffuse, specular, falloff) +vec3 lightTerms(vec3 normal, vec3 pos) { + float lightDist = length(pos - u_lightPos); + float falloff = max(0.0, u_lightRad - lightDist); + + vec3 camdir = normalize(u_cameraPos - pos); + vec3 lightdir = normalize(u_lightPos - pos); + + float diffuseTerm = clamp(dot(normal, lightdir), 0.0, 1.0); + vec3 H_L = normalize(lightdir + camdir); + float specularRV = clamp(dot(normal, H_L), 0.0, 1.0); + float specularTerm = pow(specularRV, 10.0); + + return vec3(diffuseTerm, specularTerm, falloff); +} + +float maxDepth(float depth, vec2 v_uv) { + float u = v_uv.x; + float v = v_uv.y; + float d1 = texture2D(u_depth, vec2(u+(2./800.), v)).x; + float d2 = texture2D(u_depth, vec2(u-(2./800.), v)).x; + float d3 = texture2D(u_depth, vec2(u, (v+2./600.))).x; + float d4 = texture2D(u_depth, vec2(u, (v-2./600.))).x; + return max( + max( + max( + abs(depth-d1), abs(depth-d2)), + abs(depth-d3)), + abs(depth-d4) + ); +} + void main() { vec4 gb0 = texture2D(u_gbufs[0], v_uv); vec4 gb1 = texture2D(u_gbufs[1], v_uv); vec4 gb2 = texture2D(u_gbufs[2], v_uv); vec4 gb3 = texture2D(u_gbufs[3], v_uv); float depth = texture2D(u_depth, v_uv).x; - // TODO: Extract needed properties from the g-buffers into local variables - // If nothing was rendered to this pixel, set alpha to 0 so that the - // postprocessing step can render the sky color. + // worldspace positions + vec3 pos = vec3(gb0); + // unlit surface color + vec3 color = vec3(gb2); + + // geometry normals + vec3 geomnor = vec3(gb1); + // normal map + vec3 normap = vec3(gb3); + // final normals + vec3 normal = applyNormalMap(geomnor, normap); + if (depth == 1.0) { - gl_FragColor = vec4(0, 0, 0, 0); + gl_FragColor = vec4(0); return; } - gl_FragColor = vec4(0, 0, 1, 1); // TODO: perform lighting calculations + vec3 terms = lightTerms(normal, pos); + float diff = terms.x; + float spec = terms.y; + float fall = terms.z; + + float lightColTotal = diff + spec; + if (u_toon == 1) { + lightColTotal = float(int(lightColTotal * TOON_STEPS)) / TOON_STEPS; + float surroundingDepth = maxDepth(depth, v_uv); + if (surroundingDepth > .01) { + gl_FragColor = vec4(0, 0, 0, 1); + return; + } + } + + vec3 litColor = 0.4 * fall * color * u_lightCol * lightColTotal; + gl_FragColor = vec4(litColor, 1); } diff --git a/glsl/deferred/debug.frag.glsl b/glsl/deferred/debug.frag.glsl index 9cbfae4..ffa5996 100644 --- a/glsl/deferred/debug.frag.glsl +++ b/glsl/deferred/debug.frag.glsl @@ -26,13 +26,18 @@ void main() { vec4 gb2 = texture2D(u_gbufs[2], v_uv); vec4 gb3 = texture2D(u_gbufs[3], v_uv); float depth = texture2D(u_depth, v_uv).x; - // TODO: Extract needed properties from the g-buffers into local variables - // These definitions are suggested for starting out, but you will probably want to change them. - vec3 pos; // World-space position - vec3 geomnor; // Normals of the geometry as defined, without normal mapping - vec3 colmap; // The color map - unlit "albedo" (surface color) - vec3 normap; // The raw normal map (normals relative to the surface they're on) - vec3 nor; // The true normals as we want to light them - with the normal map applied to the geometry normals (applyNormalMap above) + + // worldspace positions + vec3 pos = vec3(gb0); + // unlit surface color + vec3 colmap = vec3(gb2); + + // geometry normals + vec3 geomnor = vec3(gb1); + // normal map + vec3 normap = vec3(gb3); + // final normals + vec3 normal = applyNormalMap(geomnor, normap); if (u_debug == 0) { gl_FragColor = vec4(vec3(depth), 1.0); @@ -45,7 +50,7 @@ void main() { } else if (u_debug == 4) { gl_FragColor = vec4(normap, 1.0); } else if (u_debug == 5) { - gl_FragColor = vec4(abs(nor), 1.0); + gl_FragColor = vec4(abs(normal), 1.0); } else { gl_FragColor = vec4(1, 0, 1, 1); } diff --git a/glsl/deferred/scissor.frag.glsl b/glsl/deferred/scissor.frag.glsl new file mode 100644 index 0000000..e84c2ce --- /dev/null +++ b/glsl/deferred/scissor.frag.glsl @@ -0,0 +1,9 @@ +#version 100 +precision highp float; +precision highp int; + +uniform vec3 u_lightCol; + +void main() { + gl_FragColor = vec4(u_lightCol * .05, 1); +} diff --git a/glsl/deferred/tile.frag.glsl b/glsl/deferred/tile.frag.glsl new file mode 100644 index 0000000..7deeaaa --- /dev/null +++ b/glsl/deferred/tile.frag.glsl @@ -0,0 +1,154 @@ +#version 100 +precision highp float; +precision highp int; + +#define NUM_GBUFFERS 4 + +uniform int u_toon; +uniform int u_debug; +uniform int u_watercolor; + +uniform vec3 u_cameraPos; +uniform sampler2D u_gbufs[NUM_GBUFFERS]; +uniform sampler2D u_depth; + +uniform sampler2D u_lightsPR; +uniform sampler2D u_lightsC; + +uniform sampler2D u_lightIndices; +uniform sampler2D u_tileOffsets; + +uniform float u_tileIdx; +uniform vec2 u_lightStep; + +const int c_maxLights = 200; + +varying vec2 v_uv; + +const float TOON_STEPS = 3.0; + +vec3 applyNormalMap(vec3 geomnor, vec3 normap) { + normap = normap * 2.0 - 1.0; + vec3 up = normalize(vec3(0.001, 1, 0.001)); + vec3 surftan = normalize(cross(geomnor, up)); + vec3 surfbinor = cross(geomnor, surftan); + return normap.y * surftan + normap.x * surfbinor + normap.z * geomnor; +} + +// vec3(diffuse, specular, falloff) +vec3 lightTerms(vec3 normal, vec3 pos, vec3 lightPos, float lightRad) { + float lightDist = length(pos - lightPos); + float falloff = max(0.0, lightRad - lightDist); + + vec3 camdir = normalize(u_cameraPos - pos); + vec3 lightdir = normalize(lightPos - pos); + + float diffuseTerm = clamp(dot(normal, lightdir), 0.0, 1.0); + vec3 H_L = normalize(lightdir + camdir); + float specularRV = clamp(dot(normal, H_L), 0.0, 1.0); + float specularTerm = pow(specularRV, 10.0); + + return vec3(diffuseTerm, specularTerm, falloff); +} + +float maxDepth(float depth, vec2 v_uv) { + float u = v_uv.x; + float v = v_uv.y; + float toon_width = 2.0; + float c_img_height = 600.0; + float c_img_width = 800.0; + float d1 = texture2D(u_depth, vec2(u+(toon_width/c_img_width), v)).x; + float d2 = texture2D(u_depth, vec2(u-(toon_width/c_img_width), v)).x; + float d3 = texture2D(u_depth, vec2(u, (v+toon_width/c_img_height))).x; + float d4 = texture2D(u_depth, vec2(u, (v-toon_width/c_img_height))).x; + return max( + max( + max( + abs(depth-d1), abs(depth-d2)), + abs(depth-d3)), + abs(depth-d4) + ); +} + +float discretize(float f) { + return float(int(f * TOON_STEPS)) / TOON_STEPS; +} + +void main() { + vec4 gb0 = texture2D(u_gbufs[0], v_uv); + vec4 gb1 = texture2D(u_gbufs[1], v_uv); + vec4 gb2 = texture2D(u_gbufs[2], v_uv); + vec4 gb3 = texture2D(u_gbufs[3], v_uv); + float depth = texture2D(u_depth, v_uv).x; + + vec3 pos = vec3(gb0); // worldspace positions + vec3 color = vec3(gb2); // unlit surface color + vec3 geomnor = vec3(gb1); // geometry normals + vec3 normap = vec3(gb3); // normal map + vec3 normal = applyNormalMap(geomnor, normap); // final normals + + vec4 tileOffsetPair = texture2D(u_tileOffsets, vec2(u_tileIdx, 0)); + int lightCount = int(tileOffsetPair.x); // number of lights to consider + vec2 lightOffset = tileOffsetPair.yz; // index to start at + + if (u_debug == 0) { + gl_FragColor = vec4(vec3(float(lightCount) / float(c_maxLights)), 1); + return; + } + + if (depth == 1.0) { + gl_FragColor = vec4(0); + return; + } + + vec3 fullColor = vec3(0); + vec2 offsetIdx = lightOffset * u_lightStep; + + float surroundingDepth = maxDepth(depth, v_uv); + if (surroundingDepth > .005) { + gl_FragColor = vec4(0, 0, 0, 1); + return; + } + + float lastLightIdx = 0.0; + for (int i = 0; i < c_maxLights; i++) { + if (i >= lightCount) { + break; + } + float lightIdx = texture2D(u_lightIndices, offsetIdx).x; + lastLightIdx = lightIdx; + + vec4 lightPR = texture2D(u_lightsPR, vec2(lightIdx, 0)); + vec4 lightC = texture2D(u_lightsC, vec2(lightIdx, 0)); + + vec3 lightCol = vec3(lightC); + vec3 lightPos = lightPR.xyz; + float lightRad = lightPR.w; + + vec3 terms = lightTerms(normal, pos, lightPos, lightRad); + float diff = terms.x; + float spec = terms.y; + float atten = terms.z; + float lightColorTerm = diff + spec; + + if (u_toon == 1) { + lightColorTerm = discretize(lightColorTerm); + } + + vec3 lightColor = 0.15 * atten * color * lightCol * (diff + spec); + + fullColor += lightColor; + offsetIdx.x += u_lightStep.x; + if (offsetIdx.x >= 1.0) { + offsetIdx.x = 0.0; + offsetIdx.y += u_lightStep.y; + } + } + + if (u_watercolor == 1) { + fullColor = vec3(discretize(fullColor.x), + discretize(fullColor.y), + discretize(fullColor.z)); + } + gl_FragColor = vec4(fullColor, 1); +} diff --git a/img/chart_opts.png b/img/chart_opts.png new file mode 100644 index 0000000..bf2d529 Binary files /dev/null and b/img/chart_opts.png differ diff --git a/img/chart_tile_size.png b/img/chart_tile_size.png new file mode 100644 index 0000000..199eddb Binary files /dev/null and b/img/chart_tile_size.png differ diff --git a/img/chart_toon.png b/img/chart_toon.png new file mode 100644 index 0000000..8be582d Binary files /dev/null and b/img/chart_toon.png differ diff --git a/img/normal.png b/img/normal.png new file mode 100644 index 0000000..09fe569 Binary files /dev/null and b/img/normal.png differ diff --git a/img/thumb.png b/img/thumb.png index 9ec8ed0..30a9f9e 100644 Binary files a/img/thumb.png and b/img/thumb.png differ diff --git a/img/tiled_textures.png b/img/tiled_textures.png new file mode 100644 index 0000000..45a26e2 Binary files /dev/null and b/img/tiled_textures.png differ diff --git a/img/toon.png b/img/toon.png new file mode 100644 index 0000000..bde80eb Binary files /dev/null and b/img/toon.png differ diff --git a/img/watercolor.png b/img/watercolor.png new file mode 100644 index 0000000..bc0f48c Binary files /dev/null and b/img/watercolor.png differ diff --git a/img/zoomed-325-lights.png b/img/zoomed-325-lights.png new file mode 100644 index 0000000..55c868b Binary files /dev/null and b/img/zoomed-325-lights.png differ diff --git a/index.html b/index.html index 9f8639c..840fd2d 100644 --- a/index.html +++ b/index.html @@ -96,7 +96,7 @@ (Disable before measuring performance.)
- +
diff --git a/js/deferredRender.js b/js/deferredRender.js index b1f238b..56fc847 100644 --- a/js/deferredRender.js +++ b/js/deferredRender.js @@ -2,6 +2,57 @@ 'use strict'; // deferredSetup.js must be loaded first + var drawScene = function(state) { + for (var i = 0; i < state.models.length; i++) { + var m = state.models[i]; + + // If you want to render one model many times, note: + // readyModelForDraw only needs to be called once. + readyModelForDraw(R.progCopy, m); + + drawReadyModel(m); + } + }; + + var bindTexturesForLightPass = function(prog) { + gl.useProgram(prog.prog); + + // * Bind all of the g-buffers and depth buffer as texture uniform + // inputs to the shader + for (var i = 0; i < R.NUM_GBUFFERS; i++) { + gl.activeTexture(gl['TEXTURE' + i]); + gl.bindTexture(gl.TEXTURE_2D, R.pass_copy.gbufs[i]); + gl.uniform1i(prog.u_gbufs[i], i); + } + gl.activeTexture(gl['TEXTURE' + R.NUM_GBUFFERS]); + gl.bindTexture(gl.TEXTURE_2D, R.pass_copy.depthTex); + gl.uniform1i(prog.u_depth, R.NUM_GBUFFERS); + }; + + var bindTexturePR = function(prog, tex, data, textureID, size) { + gl.activeTexture(gl['TEXTURE' + textureID]); + gl.bindTexture(gl.TEXTURE_2D, tex); + gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, size, 1, 0, gl.RGBA, gl.FLOAT, data); + + gl.uniform1i(prog.u_lightsPR, textureID); + }; + + var bindTextureRGB = function(prog, uniformTarget, tex, data, textureID, size) { + gl.activeTexture(gl['TEXTURE' + textureID]); + gl.bindTexture(gl.TEXTURE_2D, tex); + gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGB, size, 1, 0, gl.RGB, gl.FLOAT, data); + + gl.uniform1i(uniformTarget, textureID); + }; + + var bindTextureLum = function(prog, uniformTarget, tex, data, textureID, xsize, ysize) { + gl.activeTexture(gl['TEXTURE' + textureID]); + gl.bindTexture(gl.TEXTURE_2D, tex); + gl.texImage2D(gl.TEXTURE_2D, 0, gl.LUMINANCE, xsize, ysize, 0, gl.LUMINANCE, gl.FLOAT, data); + + gl.uniform1i(uniformTarget, textureID); + }; + R.deferredRender = function(state) { if (!aborted && ( !R.progCopy || @@ -9,45 +60,39 @@ !R.progClear || !R.prog_Ambient || !R.prog_BlinnPhong_PointLight || - !R.prog_Debug || - !R.progPost1)) { + !R.progScissor || + !R.progTiled || + !R.prog_Debug || !R.progPost1)) { console.log('waiting for programs to load...'); return; } // Move the R.lights - for (var i = 0; i < R.lights.length; i++) { - // OPTIONAL TODO: Edit if you want to change how lights move - var mn = R.light_min[1]; - var mx = R.light_max[1]; - R.lights[i].pos[1] = (R.lights[i].pos[1] + R.light_dt - mn + mx) % mx + mn; + if (cfg.movingLights) { + for (var i = 0; i < R.lights.length; i++) { + var mn = R.light_min[1]; + var mx = R.light_max[1]; + R.lights[i].pos[1] = (R.lights[i].pos[1] + R.light_dt - mn + mx) % mx + mn; + } } - // Execute deferred shading pipeline - - // CHECKITOUT: START HERE! You can even uncomment this: - //debugger; - - { // TODO: this block should be removed after testing renderFullScreenQuad - gl.bindFramebuffer(gl.FRAMEBUFFER, null); - // TODO: Implement/test renderFullScreenQuad first - renderFullScreenQuad(R.progRed); - return; - } + // Update light textures with new position values + R.writeLightTextures(); + // Execute deferred shading pipeline R.pass_copy.render(state); if (cfg && cfg.debugView >= 0) { // Do a debug render instead of a regular render // Don't do any post-processing in debug mode R.pass_debug.render(state); + } else if (cfg.optimization == 1) { + R.pass_tiled.render(state); + R.pass_post1.render(state); } else { - // * Deferred pass and postprocessing pass(es) - // TODO: uncomment these - //R.pass_deferred.render(state); - //R.pass_post1.render(state); - - // OPTIONAL TODO: call more postprocessing passes, if any + // both unoptimized deferred and scissor pass through here + R.pass_deferred.render(state); + R.pass_post1.render(state); } }; @@ -56,39 +101,26 @@ */ R.pass_copy.render = function(state) { // * Bind the framebuffer R.pass_copy.fbo - // TODO: ^ + gl.bindFramebuffer(gl.FRAMEBUFFER, R.pass_copy.fbo); // * Clear screen using R.progClear - TODO: renderFullScreenQuad(R.progClear); + renderFullScreenQuad(R.progClear); // * Clear depth buffer to value 1.0 using gl.clearDepth and gl.clear - // TODO: ^ - // TODO: ^ + gl.clearDepth(1.0); + gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT); // * "Use" the program R.progCopy.prog - // TODO: ^ - // TODO: Write glsl/copy.frag.glsl + gl.useProgram(R.progCopy.prog); var m = state.cameraMat.elements; // * Upload the camera matrix m to the uniform R.progCopy.u_cameraMat // using gl.uniformMatrix4fv - // TODO: ^ + gl.uniformMatrix4fv(R.progCopy.u_cameraMat, false, m); // * Draw the scene drawScene(state); }; - var drawScene = function(state) { - for (var i = 0; i < state.models.length; i++) { - var m = state.models[i]; - - // If you want to render one model many times, note: - // readyModelForDraw only needs to be called once. - readyModelForDraw(R.progCopy, m); - - drawReadyModel(m); - } - }; - R.pass_debug.render = function(state) { // * Unbind any framebuffer, so we can write to the screen gl.bindFramebuffer(gl.FRAMEBUFFER, null); @@ -102,6 +134,133 @@ renderFullScreenQuad(R.prog_Debug); }; + /** + * 'tiled' pass: Add lighting results for each individual light + */ + R.pass_tiled.render = function(state) { + // * Bind R.pass_deferred.fbo to write into for later postprocessing + gl.bindFramebuffer(gl.FRAMEBUFFER, R.pass_deferred.fbo); + + // * Clear depth to 1.0 and color to black + gl.clearColor(0.0, 0.0, 0.0, 0.0); + gl.clearDepth(1.0); + gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT); + + // * Bind/setup the ambient pass, and render using fullscreen quad + bindTexturesForLightPass(R.prog_Ambient); + gl.uniform1f(R.prog_Ambient.u_ambientTerm, cfg.ambient); + renderFullScreenQuad(R.prog_Ambient); + + // Constants + var TILE_SIZE = cfg.tileSize; + var TILES_WIDTH = Math.ceil((width+1) / TILE_SIZE); + var TILES_HEIGHT = Math.ceil((height+1) / TILE_SIZE); + var NUM_TILES = TILES_WIDTH * TILES_HEIGHT; + + // [ tiles ] [ lights per tile ]. + var tileLights = []; + + // Create an inner array for each outer array. + for (var i = 0; i < TILES_WIDTH * TILES_HEIGHT; i++) { + tileLights.push([]); + } + + // Store lights into tileLights. + for (var lightIdx = 0; lightIdx < R.lights.length; lightIdx++) { + var light = R.lights[lightIdx]; + var lightIdxStore = (lightIdx + 0.5) / R.lights.length; + var sc = getScissorForLight(state.viewMat, state.projMat, light); + // xmin, ymin, xwidth, ywidth + if (sc !== null && sc[2] > 0 && sc[3] > 0) { + var tileX = Math.floor(sc[0] / TILE_SIZE); + var tileY = Math.floor(sc[1] / TILE_SIZE); + var tileW = Math.ceil (sc[2] / TILE_SIZE)+1; + var tileH = Math.ceil (sc[3] / TILE_SIZE)+1; + + for (var y = tileY; y < tileY + tileH; y++) { + for (var x = tileX; x < tileX + tileW; x++) { + var idx = x + (TILES_HEIGHT - 1 - y) * TILES_WIDTH; + if (idx >= 0 && idx < TILES_WIDTH * TILES_HEIGHT) { + tileLights[idx].push(lightIdxStore); + } + } + } + } else { + continue; + } + } + + // Generate textures from tileLights. + var lightIndices = new Float32Array(R.MAX_LIGHTS * NUM_TILES); + var tileOffsets = new Float32Array(3 * NUM_TILES); + + // Loop over tiles + var totalOffset = 0; + for (var tileIdx = 0; tileIdx < TILES_WIDTH * TILES_HEIGHT; tileIdx++) { + var lights = tileLights[tileIdx]; + var len = lights.length; + + tileOffsets[3*tileIdx] = len + 0.5; + tileOffsets[3*tileIdx+1] = (totalOffset % 4096) + 0.5; + tileOffsets[3*tileIdx+2] = Math.floor(totalOffset / 4096) + 0.5; + + for (lightIdx = 0; lightIdx < Math.min(len, R.MAX_LIGHTS); lightIdx++) { + lightIndices[totalOffset] = lights[lightIdx]; + totalOffset++; + } + } + + // Enable blending and scissor testing + gl.enable(gl.BLEND); + gl.blendFunc(gl.ONE, gl.ONE); + gl.enable(gl.SCISSOR_TEST); + + // Bind/setup the tiled program and uniforms that don't change. + var program = R.progTiled; + bindTexturesForLightPass(program); + + var cam = state.cameraPos; + gl.uniform3f(program.u_cameraPos, cam.x, cam.y, cam.z); + gl.uniform1i(program.u_toon, cfg.toon ? 1 : 0); + gl.uniform1i(program.u_watercolor, cfg.watercolor ? 1 : 0); + + bindTexturePR(program, + R.pass_tiled.lightDataPosRad, R.lightTexturePosRad, + R.NUM_GBUFFERS+1, R.lights.length); + + bindTextureRGB(program, program.u_lightsC, + R.pass_tiled.lightDataCol, R.lightTextureCol, + R.NUM_GBUFFERS+2, R.lights.length); + + var xsize = 4096; + var ysize = Math.ceil(totalOffset / 4096); + + bindTextureLum(program, program.u_lightIndices, + R.pass_tiled.lightTileTex, lightIndices, + R.NUM_GBUFFERS+3, xsize, ysize); + + gl.uniform2f(program.u_lightStep, 1 / xsize, 1 / ysize); + gl.uniform1i(program.u_debugView, cfg.tileDebugView); + + bindTextureRGB(program, program.u_tileOffsets, + R.pass_tiled.tileOffsetTex, tileOffsets, + R.NUM_GBUFFERS+4, tileOffsets.length / 3); + + // Loop through the tiles and call the program for each. + for (var x = 0; x < TILES_WIDTH; x++) { + for (var y = 0; y < TILES_HEIGHT; y++) { + gl.scissor(x * TILE_SIZE, y * TILE_SIZE, TILE_SIZE, TILE_SIZE); + var idx = x + (TILES_HEIGHT - 1 - y) * TILES_WIDTH; + gl.uniform1f(program.u_tileIdx, (idx + 0.5) / NUM_TILES); + renderFullScreenQuad(program); + } + } + + // Disable gl features + gl.disable(gl.SCISSOR_TEST); + gl.disable(gl.BLEND); + }; + /** * 'deferred' pass: Add lighting results for each individual light */ @@ -118,46 +277,53 @@ // Enable blending and use gl.blendFunc to blend with: // color = 1 * src_color + 1 * dst_color - // TODO: ^ + gl.enable(gl.BLEND); + gl.blendFunc(gl.ONE, gl.ONE); // * Bind/setup the ambient pass, and render using fullscreen quad bindTexturesForLightPass(R.prog_Ambient); + gl.uniform1f(R.prog_Ambient.u_ambientTerm, cfg.ambient); renderFullScreenQuad(R.prog_Ambient); - // * Bind/setup the Blinn-Phong pass, and render using fullscreen quad - bindTexturesForLightPass(R.prog_BlinnPhong_PointLight); + if (cfg.optimization == 0) { + gl.enable(gl.SCISSOR_TEST); + } else { + } - // TODO: add a loop here, over the values in R.lights, which sets the - // uniforms R.prog_BlinnPhong_PointLight.u_lightPos/Col/Rad etc., - // then does renderFullScreenQuad(R.prog_BlinnPhong_PointLight). + // * Bind/setup the Blinn-Phong pass, and render using fullscreen quad + var cam = state.cameraPos; + var program = cfg.debugScissor ? R.progScissor : R.prog_BlinnPhong_PointLight; + bindTexturesForLightPass(program); + gl.uniform1i(program.u_toon, cfg.toon ? 1 : 0); + for (var i = 0; i < R.lights.length; i++) { + var light = R.lights[i]; + var sc = getScissorForLight(state.viewMat, state.projMat, light); + if (sc !== null && sc[2] > 0 && sc[3] > 0) { + gl.scissor(sc[0], sc[1], sc[2], sc[3]); + } else { + continue; + } - // TODO: In the lighting loop, use the scissor test optimization - // Enable gl.SCISSOR_TEST, render all lights, then disable it. - // - // getScissorForLight returns null if the scissor is off the screen. - // Otherwise, it returns an array [xmin, ymin, width, height]. - // - // var sc = getScissorForLight(state.viewMat, state.projMat, light); + if (cfg.debugScissor) { + gl.uniform3f(program.u_lightCol, + light.col[0], light.col[1], light.col[2]); + } else { + gl.uniform3f(program.u_cameraPos, + cam.x, cam.y, cam.z); + gl.uniform3f(program.u_lightCol, + light.col[0], light.col[1], light.col[2]); + gl.uniform3f(program.u_lightPos, + light.pos[0], light.pos[1], light.pos[2]); + gl.uniform1f(program.u_lightRad, cfg.lightRadius); + } + renderFullScreenQuad(program); + } + gl.disable(gl.SCISSOR_TEST); // Disable blending so that it doesn't affect other code gl.disable(gl.BLEND); }; - var bindTexturesForLightPass = function(prog) { - gl.useProgram(prog.prog); - - // * Bind all of the g-buffers and depth buffer as texture uniform - // inputs to the shader - for (var i = 0; i < R.NUM_GBUFFERS; i++) { - gl.activeTexture(gl['TEXTURE' + i]); - gl.bindTexture(gl.TEXTURE_2D, R.pass_copy.gbufs[i]); - gl.uniform1i(prog.u_gbufs[i], i); - } - gl.activeTexture(gl['TEXTURE' + R.NUM_GBUFFERS]); - gl.bindTexture(gl.TEXTURE_2D, R.pass_copy.depthTex); - gl.uniform1i(prog.u_depth, R.NUM_GBUFFERS); - }; - /** * 'post1' pass: Perform (first) pass of post-processing */ @@ -174,9 +340,9 @@ // * Bind the deferred pass's color output as a texture input // Set gl.TEXTURE0 as the gl.activeTexture unit - // TODO: ^ + gl.activeTexture(gl.TEXTURE0); // Bind the TEXTURE_2D, R.pass_deferred.colorTex to the active texture unit - // TODO: ^ + gl.bindTexture(gl.TEXTURE_2D, R.pass_deferred.colorTex); // Configure the R.progPost1.u_color uniform to point at texture unit 0 gl.uniform1i(R.progPost1.u_color, 0); @@ -193,24 +359,19 @@ // vertex shader doesn't have to do any transformation; draw two // triangles which cover the screen over x = -1..1 and y = -1..1. // This array is set up to use gl.drawArrays with gl.TRIANGLE_STRIP. - var positions = new Float32Array([ - -1.0, -1.0, 0.0, - 1.0, -1.0, 0.0, - -1.0, 1.0, 0.0, - 1.0, 1.0, 0.0 - ]); + var positions = R.quadPositions; var vbo = null; var init = function() { // Create a new buffer with gl.createBuffer, and save it as vbo. - // TODO: ^ + vbo = gl.createBuffer(); // Bind the VBO as the gl.ARRAY_BUFFER - // TODO: ^ + gl.bindBuffer(gl.ARRAY_BUFFER, vbo); // Upload the positions array to the currently-bound array buffer // using gl.bufferData in static draw mode. - // TODO: ^ + gl.bufferData(gl.ARRAY_BUFFER, positions, gl.STATIC_DRAW); }; return function(prog) { @@ -223,16 +384,18 @@ gl.useProgram(prog.prog); // Bind the VBO as the gl.ARRAY_BUFFER - // TODO: ^ + gl.bindBuffer(gl.ARRAY_BUFFER, vbo); + // Enable the bound buffer as the vertex attrib array for // prog.a_position, using gl.enableVertexAttribArray - // TODO: ^ + gl.enableVertexAttribArray(vbo); + // Use gl.vertexAttribPointer to tell WebGL the type/layout for // prog.a_position's access pattern. - // TODO: ^ + gl.vertexAttribPointer(vbo, 3, gl.FLOAT, false, 0, 0); // Use gl.drawArrays (or gl.drawElements) to draw your quad. - // TODO: ^ + gl.drawArrays(gl.TRIANGLE_STRIP, 0, 4); // Unbind the array buffer. gl.bindBuffer(gl.ARRAY_BUFFER, null); diff --git a/js/deferredSetup.js b/js/deferredSetup.js index 115b016..306f354 100644 --- a/js/deferredSetup.js +++ b/js/deferredSetup.js @@ -5,8 +5,18 @@ R.pass_copy = {}; R.pass_debug = {}; R.pass_deferred = {}; + R.pass_tiled = {}; R.pass_post1 = {}; R.lights = []; + R.lightTexturePosRad = new Float32Array(); + R.lightTextureCol = new Float32Array(); + R.quadPositions = new Float32Array([ + -1.0, -1.0, 0.0, + 1.0, -1.0, 0.0, + -1.0, 1.0, 0.0, + 1.0, 1.0, 0.0 + ]); + R.lightIndices = new Float32Array(100); R.NUM_GBUFFERS = 4; @@ -14,19 +24,23 @@ * Set up the deferred pipeline framebuffer objects and textures. */ R.deferredSetup = function() { - setupLights(); + R.setupLights(R.NUM_LIGHTS, R.LIGHT_RADIUS, 48); loadAllShaderPrograms(); R.pass_copy.setup(); R.pass_deferred.setup(); + R.pass_tiled.setup(); }; - // TODO: Edit if you want to change the light initial positions - R.light_min = [-14, 0, -6]; - R.light_max = [14, 18, 6]; + R.light_min = [-14, 0, -4]; + R.light_max = [14, 18, 4]; R.light_dt = -0.03; + + // defaults R.LIGHT_RADIUS = 4.0; - R.NUM_LIGHTS = 20; // TODO: test with MORE lights! - var setupLights = function() { + R.NUM_LIGHTS = 20; + R.MAX_LIGHTS = 200; + + R.setupLights = function(numLights, lightRadius, numTiles) { Math.seedrandom(0); var posfn = function() { @@ -39,16 +53,50 @@ return r; }; - for (var i = 0; i < R.NUM_LIGHTS; i++) { + for (var i = 0; i < R.lights.length; i++) { + R.lights[i].rad = lightRadius + (Math.random() - 0.5); + } + // Add more lights if adding lights + for (i = R.lights.length; i < numLights; i++) { R.lights.push({ pos: posfn(), col: [ 1 + Math.random(), 1 + Math.random(), 1 + Math.random()], - rad: R.LIGHT_RADIUS + rad: lightRadius + (Math.random() - 0.5) }); } + // And slice to size, if you're removing lights + R.lights.length = numLights; + //R.lights = R.lights.slice(0, numLights); + + R.lightTexturePosRad = new Float32Array(4 * numLights); + R.lightTextureCol = new Float32Array(3 * numLights); + + R.lightIndices = new Float32Array(R.MAX_LIGHTS, numTiles); + + // Store colors + for (i = 0; i < numLights; i++) { + var light = R.lights[i]; + var idxC = 3*i; + R.lightTextureCol[idxC+0] = light.col[0]; + R.lightTextureCol[idxC+1] = light.col[1]; + R.lightTextureCol[idxC+2] = light.col[2]; + } + R.writeLightTextures(); + }; + + // Write new position values values into R.lightTexturePosRad. + R.writeLightTextures = function() { + for (var i = 0; i < R.lights.length; i++) { + var light = R.lights[i]; + var idxPR = 4*i; + R.lightTexturePosRad[idxPR+0] = light.pos[0]; + R.lightTexturePosRad[idxPR+1] = light.pos[1]; + R.lightTexturePosRad[idxPR+2] = light.pos[2]; + R.lightTexturePosRad[idxPR+3] = light.rad; + } }; /** @@ -94,6 +142,23 @@ gl_draw_buffers.drawBuffersWEBGL([gl_draw_buffers.COLOR_ATTACHMENT0_WEBGL]); }; + /** + * Create/configure framebuffer between "deferred" and "post1" stages + */ + R.pass_tiled.setup = function() { + // Don't create another FBO -- this will actually write into + // pass_deferred, so the post-processing step can share that FBO. + + // contains light (pos, radius) data + R.pass_tiled.lightDataPosRad = createAndBindLightDataTexture(); + // contains light (color) data + R.pass_tiled.lightDataCol = createAndBindLightDataTexture(); + // contains lights per tile + R.pass_tiled.lightTileTex = createAndBindLightDataTexture(); + // contains indices into lightTileTex + R.pass_tiled.tileOffsetTex = createAndBindLightDataTexture(); + }; + /** * Loads all of the shader programs used in the pipeline. */ @@ -129,17 +194,44 @@ loadDeferredProgram('ambient', function(p) { // Save the object into this variable for access later + p.u_ambientTerm = gl.getUniformLocation(p.prog, 'u_ambientTerm'); R.prog_Ambient = p; }); loadDeferredProgram('blinnphong-pointlight', function(p) { // Save the object into this variable for access later + p.u_cameraPos = gl.getUniformLocation(p.prog, 'u_cameraPos'); p.u_lightPos = gl.getUniformLocation(p.prog, 'u_lightPos'); p.u_lightCol = gl.getUniformLocation(p.prog, 'u_lightCol'); p.u_lightRad = gl.getUniformLocation(p.prog, 'u_lightRad'); + p.u_toon = gl.getUniformLocation(p.prog, 'u_toon'); R.prog_BlinnPhong_PointLight = p; }); + loadDeferredProgram('tile', function(p) { + // Save the object into this variable for access later + p.u_cameraPos = gl.getUniformLocation(p.prog, 'u_cameraPos'); + p.u_toon = gl.getUniformLocation(p.prog, 'u_toon'); + p.u_watercolor = gl.getUniformLocation(p.prog, 'u_watercolor'); + p.u_debugView = gl.getUniformLocation(p.prog, 'u_debug'); + + p.u_lightsPR = gl.getUniformLocation(p.prog, 'u_lightsPR'); + p.u_lightsC = gl.getUniformLocation(p.prog, 'u_lightsC'); + + p.u_lightIndices = gl.getUniformLocation(p.prog, 'u_lightIndices'); + p.u_tileOffsets = gl.getUniformLocation(p.prog, 'u_tileOffsets'); + + p.u_tileIdx = gl.getUniformLocation(p.prog, 'u_tileIdx'); + p.u_lightStep = gl.getUniformLocation(p.prog, 'u_lightStep'); + R.progTiled = p; + }); + + loadDeferredProgram('scissor', function(p) { + // Save the object into this variable for access later + p.u_lightCol = gl.getUniformLocation(p.prog, 'u_lightCol'); + R.progScissor = p; + }); + loadDeferredProgram('debug', function(p) { p.u_debug = gl.getUniformLocation(p.prog, 'u_debug'); // Save the object into this variable for access later @@ -222,4 +314,16 @@ return tex; }; + + var createAndBindLightDataTexture = function() { + var tex = gl.createTexture(); + gl.bindTexture(gl.TEXTURE_2D, tex); + gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST); + gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST); + gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE); + gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE); + //gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, width, height, 0, gl.RGBA, gl.FLOAT, null); + gl.bindTexture(gl.TEXTURE_2D, null); + return tex; + }; })(); diff --git a/js/framework.js b/js/framework.js index bd07d47..942f582 100644 --- a/js/framework.js +++ b/js/framework.js @@ -68,9 +68,17 @@ var width, height; var init = function() { // TODO: For performance measurements, disable debug mode! - var debugMode = true; + var debugMode = false; + + canvas = document.getElementById('canvas'); + renderer = new THREE.WebGLRenderer({ + canvas: canvas, + preserveDrawingBuffer: debugMode + }); + gl = renderer.context; if (debugMode) { + $('#dlbutton button').attr('disabled', false); $('#debugmodewarning').css('display', 'block'); var throwOnGLError = function(err, funcName, args) { abort(WebGLDebugUtils.glEnumToString(err) + @@ -79,13 +87,6 @@ var width, height; gl = WebGLDebugUtils.makeDebugContext(gl, throwOnGLError); } - canvas = document.getElementById('canvas'); - renderer = new THREE.WebGLRenderer({ - canvas: canvas, - preserveDrawingBuffer: debugMode - }); - gl = renderer.context; - initExtensions(); stats = new Stats(); diff --git a/js/ui.js b/js/ui.js index 05c1852..5f322eb 100644 --- a/js/ui.js +++ b/js/ui.js @@ -4,31 +4,78 @@ var cfg; 'use strict'; var Cfg = function() { - // TODO: Define config fields and defaults here this.debugView = -1; this.debugScissor = false; - this.enableEffect0 = false; + + this.optimization = 1; + this.movingLights = true; + this.toon = false; + this.watercolor = false; + + this.ambient = 0.1; + this.lightRadius = 4.0; + this.numLights = 50; + + this.tileSize = 100; + this.tileDebugView = -1; }; var init = function() { cfg = new Cfg(); var gui = new dat.GUI(); - // TODO: Define any other possible config values - gui.add(cfg, 'debugView', { - 'None': -1, - '0 Depth': 0, - '1 Position': 1, - '2 Geometry normal': 2, - '3 Color map': 3, - '4 Normal map': 4, - '5 Surface normal': 5 + var debug = gui.addFolder('Debug Views'); + debug.add(cfg, 'debugView', { + 'None': -1, + '[0] Depth': 0, + '[1] Position': 1, + '[2] Geometry normal': 2, + '[3] Color map': 3, + '[4] Normal map': 4, + '[5] Surface normal': 5 + }); + debug.open(); + + var opt = gui.addFolder('Optimizations'); + opt.add(cfg, 'optimization', { + 'None': -1, + 'Scissor': 0, + 'Tile': 1, + }); + opt.add(cfg, 'debugScissor'); + opt.open(); + + var effects = gui.addFolder('Effects'); + effects.add(cfg, 'movingLights'); + effects.add(cfg, 'toon'); + effects.add(cfg, 'watercolor'); + effects.open(); + + var updateLights = function() { + var TILE_SIZE = cfg.tileSize; + var TILES_WIDTH = Math.ceil((width+1) / TILE_SIZE); + var TILES_HEIGHT = Math.ceil((height+1) / TILE_SIZE); + var NUM_TILES = TILES_WIDTH * TILES_HEIGHT; + R.setupLights(cfg.numLights, cfg.lightRadius, NUM_TILES); + }; + + var consts = gui.addFolder('Constants'); + consts.add(cfg, 'ambient', 0.1, 1.0); + consts.add(cfg, 'lightRadius', 0.5, 10.0).onFinishChange(updateLights); + consts.add(cfg, 'numLights').min(5).max(500).step(5).onFinishChange(updateLights); + //consts.add(cfg, 'numLights').min(50).max(100).step(10).onFinishChange(updateLights); + + consts.open(); + + var tileOpts = gui.addFolder('Tile Options'); + tileOpts.add(cfg, 'tileSize').min(10).max(150).step(25); + tileOpts.add(cfg, 'tileDebugView', { + 'None': -1, + '# Lights': 0 }); - gui.add(cfg, 'debugScissor'); + tileOpts.open(); - var eff0 = gui.addFolder('EFFECT NAME HERE'); - eff0.add(cfg, 'enableEffect0'); - // TODO: add more effects toggles and parameters here + updateLights(); }; window.handle_load.push(init);