diff --git a/README.md b/README.md index f810e88..e7dc72a 100644 --- a/README.md +++ b/README.md @@ -3,424 +3,157 @@ WebGL Deferred Shading **University of Pennsylvania, CIS 565: GPU Programming and Architecture, Project 6** -* (TODO) YOUR NAME HERE -* Tested on: (TODO) **Google Chrome 222.2** on - Windows 22, i7-2222 @ 2.22GHz 22GB, GTX 222 222MB (Moore 2222 Lab) +* Tongbo Sui +* Tested on: Google Chrome 46.0.2490.71, Firefox 41.0.2, on Windows 10, i5-3320M @ 2.60GHz 8GB, NVS 5400M 2GB (Personal) + +### Overview +WebGL based deferred lighting, with bloom and toon effects. ### Live Online -[![](img/thumb.png)](http://TODO.github.io/Project6-WebGL-Deferred-Shading) +[![](img/thumb.png)](http://stonebird.github.io/Project6-WebGL-Deferred-Shading/) ### Demo Video -[![](img/video.png)](TODO) - -### (TODO: Your README) - -*DO NOT* leave the README to the last minute! It is a crucial part of the -project, and we will not be able to grade you without a good README. - -This assignment has a considerable amount of performance analysis compared -to implementation work. Complete the implementation early to leave time! - - -Instructions (delete me) -======================== - -This is due at midnight on the evening of Tuesday, October 27. - -**Summary:** In this project, you'll be introduced to the basics of deferred -shading and WebGL. You'll use GLSL and WebGL to implement a deferred shading -pipeline and various lighting and visual effects. - -**Recommendations:** -Take screenshots as you go. Use them to document your progress in your README! - -Read (or at least skim) the full README before you begin, so that you know what -to expect and what to prepare for. - -### Running the code - -If you have Python, you should be able to run `server.py` to start a server. -Then, open [`http://localhost:10565/`](http://localhost:10565/) in your browser. +[![](img/video.png)](https://youtu.be/w9zWxD3v8TM) -This project requires a WebGL-capable web browser with support for -`WEBGL_draw_buffers`. You can check for support on -[WebGL Report](http://webglreport.com/). +### Features and Analysis -Google Chrome seems to work best on all platforms. If you have problems running -the starter code, use Chrome or Chromium, and make sure you have updated your -browser and video drivers. +#### Effects -In Moore 100C, both Chrome and Firefox work. -See below for notes on profiling/debugging tools. +* Deferred Blinn-Phong shading [4] + * Blinn-Phong lighting with normal mapping support, rendered in a separate pass outside fragment shader + * *Optimization*: remove out surface normal calculations (see Optimized g-buffer) + * *Possible further improvements*: better falloff computation -Use the screenshot button to save a screenshot. +###### Deferred render -## Requirements +![](img/deferred-render.png) -**Ask on the mailing list for any clarifications.** +###### Debug views for deferred lighting -In this project, you are given code for: +![](img/debugs.png) -* Loading OBJ files and color/normal map textures -* Camera control -* Partial implementation of deferred shading including many helper functions +* Bloom + * Post-process Gaussian blur [1] used to simulate glow effect + * *Optimization*: use two-pass Gaussian blur instead of one single convolution kernel + * Single convolution kernel requires significantly more time to compute. For a 5x5 kernel, that would require 25 reads per fragment + * On the other hand, two-pass convolution only needs 10 reads per fragment, which yields a 2.5x speedup + * *Possible further improvements*: none + * *Performance impact*: minor performance impact; see graph below -### Required Tasks +###### Bloom -**Before doing performance analysis,** you must disable debug mode by changing -`debugMode` to `false` in `framework.js`. Keep it enabled when developing - it -helps find WebGL errors *much* more easily. +![](img/bloom-shading.png) -You will need to perform the following tasks: +* Toon shading + * Ramp shading with an edge detector for rendering outlines [2][3] + * Uses Sobel operator as edge detectors. The operator requires two convolution passes, each of which is a 3x3 kernel + * *Possible further improvements*: more passes for faster convolution. E.g. break each pass further into 2 passes, so that in each pass the shader is only applying convolution with a 1D kernel + * *Performance impact*: medium performance impact; see graph below -* Complete the deferred shading pipeline so that the Blinn-Phong and Post1 - shaders recieve the correct input. Go through the Starter Code Tour **before - continuing!** +###### Toon -**Effects:** +![](img/toon-shading.png) -* Implement deferred Blinn-Phong shading (diffuse + specular) - * With normal mapping (code provided) +* Additional material properties + * Supports specular exponent as an input to g-buffers + * *Performance impact*: no visible impact -* Implement one of the following effects: - * Bloom using post-process blur (box or Gaussian) [1] - * Toon shading (with ramp shading + simple depth-edge detection for outlines) +###### Additional material property -**Optimizations:** +![](img/spec-exp.png) -* Scissor test optimization: when accumulating shading from each point - light source, only render in a rectangle around the light. - * Show a debug view for this (showing scissor masks clearly), e.g. by - modifying and using `red.frag.glsl` with additive blending and alpha = 0.1. - * Code is provided to compute this rectangle for you, and there are - comments at the relevant place in `deferredRender.js` with more guidance. +###### Frame time (ms) with varying # of lights -* Optimized g-buffer format - reduce the number and size of g-buffers: - * Ideas: - * Pack values together into vec4s - * Use 2-component normals - * Quantize values by using smaller texture types instead of gl.FLOAT - * Reduce number of properties passed via g-buffer, e.g. by: - * Applying the normal map in the `copy` shader pass instead of - copying both geometry normals and normal maps - * Reconstructing world space position using camera matrices and X/Y/depth - * For credit, you must show a good optimization effort and record the - performance of each version you test, in a simple table. - * It is expected that you won't need all 4 provided g-buffers for a basic - pipeline - make sure you disable the unused ones. - * See mainly: `copy.frag.glsl`, `deferred/*.glsl`, `deferredSetup.js` +![](img/light-time.png) -### Extra Tasks +#### Optimizations -You must do at least **10 points** worth of extra features (effects or -optimizations/analysis). +* Optimized g-buffer format, reduce the number and size of g-buffers: + * Surface normal pre-calculation + * Applying the normal map in the `copy` shader pass instead of copying both geometry normals and normal maps + * *Performance analysis* + * Significant improvement over non-optimized g-buffer for the following two reasons: + * 1 less g-buffer to read/write. This results in overall shorter memory access time + * Since the Blinn-Phong shader shades fragments in a loop, pre-calculating surface normal avoids recalculating surface normal over and over again inside the loop. This greatly reduces the computation time during deferred lighting + * Tradeoff: less flexibility inside the shader. The shader won't get geometry normal as input. + * RGBA packing; code derived from [5] + * On top of pre-calculating surface normal, packing RGBA information into a single floating point value in g-buffer + * Color channels are packed into one single float by simulating bit-shifts with multiplication + * *Performance analysis* + * Minor performance improvement, due to one less g-buffer and hence overall faster memory access + * In general, this provides only marginal performance improvement, while reducing color quality greatly during color reconstruction + * Tradeoff: significant loss in color quality + * Improved RGBA packing; same code structure of [5], but different packing method + * On top of pre-calculating surface normal, packing RGBA information into a single floating point value in g-buffer + * Instead of simulating bit-shifts, operate in base 10 and allocate each channel 3 digits in the floating point value + * Due to some unknown issue with GLSL on floating point values (either precision or interpretation), more digits per channel is not feasible + * This doesn't include alpha channel. Therefore it's a 9-digit integer stored as a floating point value. With alpha, each channel will only get 2 digits + * *Performance analysis* + * Exact same performance benefit as "old" color-packing method, but without visible color quality loss + * Color quality losses do happen. In fact, due to the ordering of the channels during packing, blue channel suffers from greater quality loss than red and green channels. + * However compared to the bit-shift method, quality loss is much less visible, if not indistinguishable + * Tradeoff + * Screenshot feature doesn't work (returns black image); Need to press `print screen` on the keyboard for anything to be captured + * Alpha channel is excluded + * `master-backup` has the version without color-packing. The render logic differs greatly in fragment shader and other shaders. Therefore they are in separate branches -**Effects:** +###### Render with RGBA packing (Old). Notice the reduced color quality of the reconstructed color map -* (3pts) The effect you didn't choose above (bloom or toon shading) +![](img/packing-demo.png) -* (3pts) Screen-space motion blur (blur along velocity direction) [3] - -* (2pts) Allow variability in additional material properties - * Include other properties (e.g. specular coeff/exponent) in g-buffers - * Use this to render objects with different material properties - * These may be uniform across one model draw call, but you'll have to show - multiple models - -**Optimizations/Analysis:** - -* (2pts) Improved screen-space AABB for scissor test - (smaller/more accurate than provided - but beware of CPU/GPU tradeoffs) - -* (3pts) Two-pass **Gaussian** blur using separable convolution (using a second - postprocess render pass) to improve bloom or other 2D blur performance - -* (4-6pts) Light proxies - * (4pts) Instead of rendering a scissored full-screen quad for every light, - render some proxy geometry which covers the part of the screen affected by - the light (e.g. a sphere, for an attenuated point light). - * A model called `sphereModel` is provided which can be drawn in the same - way as the code in `drawScene`. (Must be drawn with a vertex shader which - scales it to the light radius and translates it to the light position.) - * (+2pts) To avoid lighting geometry far behind the light, render the proxy - geometry (e.g. sphere) using an inverted depth test - (`gl.depthFunc(gl.GREATER)`) with depth writing disabled (`gl.depthMask`). - This test will pass only for parts of the screen for which the backside of - the sphere appears behind parts of the scene. - * Note that the copy pass's depth buffer must be bound to the FBO during - this operation! - * Show a debug view for this (showing light proxies) - * Compare performance of this, naive, and scissoring. - -* (8pts) Tile-based deferred shading with detailed performance comparison - * On the CPU, check which lights overlap which tiles. Then, render each tile - just once for all lights (instead of once for each light), applying only - the overlapping lights. - * The method is described very well in - [Yuqin & Sijie's README](https://github.com/YuqinShao/Tile_Based_WebGL_DeferredShader/blob/master/README.md#algorithm-details). - * This feature requires allocating the global light list and tile light - index lists as shown at this link. These can be implemented as textures. - * Show a debug view for this (number of lights per tile) - -* (6pts) Deferred shading without multiple render targets - (i.e. without WEBGL_draw_buffers). - * Render the scene once for each target g-buffer, each time into a different - framebuffer object. - * Include a detailed performance analysis, comparing with/without - WEBGL_draw_buffers (like in the - [Mozilla blog article](https://hacks.mozilla.org/2014/01/webgl-deferred-shading/)). - -* (2-6pts) Compare performance to equivalently-lit forward-rendering: - * (2pts) With no forward-rendering optimizations - * (+2pts) Coarse, per-object back-to-front sorting of geometry for early-z - * (Of course) must render many objects to test - * (+2pts) Z-prepass for early-z - -This extra feature list is not comprehensive. If you have a particular idea -that you would like to implement, please **contact us first** (preferably on -the mailing list). - -**Where possible, all features should be switchable using the GUI panel in -`ui.js`.** - -### Performance & Analysis - -**Before doing performance analysis,** you must disable debug mode by changing -`debugMode` to `false` in `framework.js`. Keep it enabled when developing - it -helps find WebGL errors *much* more easily. - -Optimize your JavaScript and/or GLSL code. Web Tracing Framework -and Chrome/Firefox's profiling tools (see Resources section) will -be useful for this. For each change -that improves performance, show the before and after render times. - -For each new *effect* feature (required or extra), please -provide the following analysis: - -* Concise overview write-up of the feature. -* Performance change due to adding the feature. - * If applicable, how do parameters (such as number of lights, etc.) - affect performance? Show data with simple graphs. -* If you did something to accelerate the feature, what did you do and why? -* How might this feature be optimized beyond your current implementation? - -For each *performance* feature (required or extra), please provide: - -* Concise overview write-up of the feature. -* Detailed performance improvement analysis of adding the feature - * What is the best case scenario for your performance improvement? What is - the worst? Explain briefly. - * Are there tradeoffs to this performance feature? Explain briefly. - * How do parameters (such as number of lights, tile size, etc.) affect - performance? Show data with graphs. - * Show debug views when possible. - * If the debug view correlates with performance, explain how. - -Note: Be aware that stats.js may give 0 millisecond frame timings in Chrome on -occasion - if this happens, you can use the FPS counter. - -### Starter Code Tour - -You'll be working mainly in `deferredRender.js` using raw WebGL. Three.js is -included in the project for various reasons. You won't use it for much, but its -matrix/vector types may come in handy. - -It's highly recommended that you use the browser debugger to inspect variables -to get familiar with the code. At any point, you can also -`console.log(some_var);` to show it in the console and inspect it. - -The setup in `deferredSetup` is already done for you, for many of the features. -If you want to add uniforms (textures or values), you'll change them here. -Therefore, it is recommended that you review the comments to understand the -process, BEFORE starting work in `deferredRender`. - -In `deferredRender`, start at the **START HERE!** comment. -Work through the appropriate `TODO`s as you go - most of them are very -small. Test incrementally (after implementing each part, instead of testing -all at once). -* (The first thing you should be doing is implementing the fullscreen quad!) - -Your _next_ first goal should be to get the debug views working. -Add code in `debug.frag.glsl` to examine your g-buffers before trying to -render them. (Set the debugView in the UI to show them.) - -For editing JavaScript, you can use a simple editor with syntax highlighting -such as Sublime, Vim, Emacs, etc., or the editor built into Chrome. - -* `js/`: JavaScript files for this project. - * `main.js`: Handles initialization of other parts of the program. - * `framework.js`: Loads the scene, camera, etc., and calls your setup/render - functions. Hopefully, you won't need to change anything here. - * `deferredSetup.js`: Deferred shading pipeline setup code. - * `createAndBind(Depth/Color)TargetTexture`: Creates empty textures for - binding to frame buffer objects as render targets. - * `deferredRender.js`: Your deferred shading pipeline execution code. - * `renderFullScreenQuad`: Renders a full-screen quad with the given shader - program. - * `ui.js`: Defines the UI using - [dat.GUI](https://workshop.chromeexperiments.com/examples/gui/). - * The global variable `cfg` can be accessed anywhere in the code to read - configuration values. - * `utils.js`: Utilities for JavaScript and WebGL. - * `abort`: Aborts the program and shows an error. - * `loadTexture`: Loads a texture from a URL into WebGL. - * `loadShaderProgram`: Loads shaders from URLs into a WebGL shader program. - * `loadModel`: Loads a model into WebGL buffers. - * `readyModelForDraw`: Configures the WebGL state to draw a model. - * `drawReadyModel`: Draws a model which has been readied. - * `getScissorForLight`: Computes an approximate scissor rectangle for a - light in world space. -* `glsl/`: GLSL code for each part of the pipeline: - * `clear.*.glsl`: Clears each of the `NUM_GBUFFERS` g-buffers. - * `copy.*.glsl`: Performs standard rendering without any fragment shading, - storing all of the resulting values into the `NUM_GBUFFERS` g-buffers. - * `quad.vert.glsl`: Minimal vertex shader for rendering a single quad. - * `deferred.frag.glsl`: Deferred shading pass (for lighting calculations). - Reads from each of the `NUM_GBUFFERS` g-buffers. - * `post1.frag.glsl`: First post-processing pass. -* `lib/`: JavaScript libraries. -* `models/`: OBJ models for testing. Sponza is the default. -* `index.html`: Main HTML page. -* `server.bat` (Windows) or `server.py` (OS X/Linux): - Runs a web server at `localhost:10565`. - -### The Deferred Shading Pipeline - -See the comments in `deferredSetup.js`/`deferredRender.js` for low-level guidance. - -In order to enable and disable effects using the GUI, upload a vec4 uniform -where each component is an enable/disable flag. In JavaScript, the state of the -UI is accessible anywhere as `cfg.enableEffect0`, etc. - -**Pass 1:** Renders the scene geometry and its properties to the g-buffers. -* `copy.vert.glsl`, `copy.frag.glsl` -* The framebuffer object `pass_copy.fbo` must be bound during this pass. -* Renders into `pass_copy.depthTex` and `pass_copy.gbufs[i]`, which need to be - attached to the framebuffer. - -**Pass 2:** Performs lighting and shading into the color buffer. -* `quad.vert.glsl`, `deferred/blinnphong-pointlight.frag.glsl` -* Takes the g-buffers `pass_copy.gbufs`/`depthTex` as texture inputs to the - fragment shader, on uniforms `u_gbufs` and `u_depth`. -* `pass_deferred.fbo` must be bound. -* Renders into `pass_deferred.colorTex`. - -**Pass 3:** Performs post-processing. -* `quad.vert.glsl`, `post/one.frag.glsl` -* Takes `pass_BlinnPhong_PointLight.colorTex` as a texture input `u_color`. -* Renders directly to the screen if there are no additional passes. - -More passes may be added for additional effects (e.g. combining bloom with -motion blur) or optimizations (e.g. two-pass Gaussian blur for bloom) - -#### Debugging - -If there is a WebGL error, it will be displayed on the developer console and -the renderer will be aborted. To find out where the error came from, look at -the backtrace of the error (you may need to click the triangle to expand the -message). The line right below `wrapper @ webgl-debug.js` will point to the -WebGL call that failed. - -#### Changing the number of g-buffers - -Note that the g-buffers are just `vec4`s - you can put any values you want into -them. However, if you want to change the total number of g-buffers (add more -for additional effects or remove some for performance), you will need to make -changes in a number of places: - -* `deferredSetup.js`/`deferredRender.js`: search for `NUM_GBUFFERS` -* `copy.frag.glsl` -* `deferred.frag.glsl` -* `clear.frag.glsl` - - -## Resources - -* [1] Bloom: - [GPU Gems, Ch. 21](http://http.developer.nvidia.com/GPUGems/gpugems_ch21.html) -* [2] Screen-Space Ambient Occlusion: - [Floored Article](http://floored.com/blog/2013/ssao-screen-space-ambient-occlusion.html) -* [3] Post-Process Motion Blur: - [GPU Gems 3, Ch. 27](http://http.developer.nvidia.com/GPUGems3/gpugems3_ch27.html) - -**Also see:** The articles linked in the course schedule. - -### Profiling and debugging tools - -Built into Firefox: -* Canvas inspector -* Shader Editor -* JavaScript debugger and profiler - -Built into Chrome: -* JavaScript debugger and profiler - -Plug-ins: -* (Chrome/Firefox) [Web Tracing Framework](http://google.github.io/tracing-framework/) -* (Chrome) [Shader Editor](https://chrome.google.com/webstore/detail/shader-editor/ggeaidddejpbakgafapihjbgdlbbbpob) - - -Firefox can also be useful - it has a canvas inspector, WebGL profiling and a -shader editor built in. +###### Quality of improved RGBA packing. Notice the almost identical color quality +![](img/packing-comparison.png) -## README +###### G-buffer optimization effects -Replace the contents of this README.md in a clear manner with the following: +![](img/gbuf-opt.png) -* A brief description of the project and the specific features you implemented. -* At least one screenshot of your project running. -* A 30+ second video of your project running showing all features. - [Open Broadcaster Software](http://obsproject.com) is recommended. - (Even though your demo can be seen online, using multiple render targets - means it won't run on many computers. A video will work everywhere.) -* A performance analysis (described below). +* Scissoring + * Scissor the screen for each light, such that it only renders in a rectangle around the light + * *Performance analysis* + * Effectivity + * Effective in most cases, as instead of calculating lighting for all fragments, only a small portion of the fragments participate in the actual computation. Hence if a fragment is lit by only a few lights, less computation is needed. + * However, it becomes ineffective when number of lights increases to a significant amount. As seen in "Frame time" graph above, at 120 lights, the frame time increases dramatically. At this point, lots of lights are covering almost every fragment in the scene. Even though for each light, it's still scissored, but since there is too much overlapping, all fragments need to participate in the computation many times. Therefore the scissoring is much less effective + * Tradeoff: some artifacts due to loose AABB calculation (fixed below) -### Performance Analysis +###### Scissoring. Light boxes are in red. Darker means less light overlaps -See above. +![](img/default-aabb-debug.png) -### GitHub Pages +* Improved screen-space AABB for scissor test + * Calculate bounding box at sphere's front-facing max circle to minimize distortion due to transformations + * *Performance analysis* + * Faster scissor tests due to more accurate (smaller) AABB calculation. Potentially much less overlapping on the edge of each light, for each fragment + * Fixes artifacts caused by default AABB calculation -Since this assignment is in WebGL, you can make your project easily viewable by -taking advantage of GitHub's project pages feature. +###### AABB size comparison. Improved AABB calculation yields smaller scissor boxes -Once you are done with the assignment, create a new branch: +![](img/aabb-compare.png) -`git branch gh-pages` +###### Scissoring performance (no effects) -Push the branch to GitHub: +![](img/scissor-pure.png) -`git push origin gh-pages` +###### Scissoring performance (effects) -Now, you can go to `.github.io/` to see your -renderer online from anywhere. Add this link to your README. +![](img/scissor-effect.png) -## Submit +* Two-pass Gaussian blur [1] + * Uses separate passes to apply convolution in both `x` and `y` directions + * *Performance analysis* + * Faster post-processing + * As stated previoiusly, single convolution kernel requires significantly more time to compute. For a 5x5 kernel, two-pass convolution yields a 2.5x speedup + * Tradeoff: one more render pass. This requires one more buffer manipulation and memory setup -1. Open a GitHub pull request so that we can see that you have finished. - The title should be "Submission: YOUR NAME". - * **ADDITIONALLY:** - In the body of the pull request, include a link to your repository. -2. Send an email to the TA (gmail: kainino1+cis565@) with: - * **Subject**: in the form of `[CIS565] Project N: PENNKEY`. - * Direct link to your pull request on GitHub. - * Estimate the amount of time you spent on the project. - * If there were any outstanding problems, briefly explain. - * **List the extra features you did.** - * Feedback on the project itself, if any. +### References -### Third-Party Code Policy +* [1] Bloom: [GPU Gems, Ch. 21](http://http.developer.nvidia.com/GPUGems/gpugems_ch21.html) +* [2] Edge detector: [Sobel Operator](https://en.wikipedia.org/wiki/Sobel_operator) +* [3] Toon ramping: [Cel shading](http://prideout.net/blog/?p=22#toon) +* [4] Blinn-Phong: [Blinn–Phong shading model](https://en.wikipedia.org/wiki/Blinn%E2%80%93Phong_shading_model) -* Use of any third-party code must be approved by asking on our mailing list. -* If it is approved, all students are welcome to use it. Generally, we approve - use of third-party code that is not a core part of the project. For example, - for the path tracer, we would approve using a third-party library for loading - models, but would not approve copying and pasting a CUDA function for doing - refraction. -* Third-party code **MUST** be credited in README.md. -* Using third-party code without its approval, including using another - student's code, is an academic integrity violation, and will, at minimum, - result in you receiving an F for the semester. +### Third-party codes +* [5] RGBA packing: [Aras Pranckevičius](http://aras-p.info/blog/2009/07/30/encoding-floats-to-rgba-the-final/) / [StackOverflow](http://stackoverflow.com/questions/30242013/glsl-compressing-packing-multiple-0-1-colours-var4-into-a-single-var4-variab) diff --git a/glsl/clear.frag.glsl b/glsl/clear.frag.glsl index b4e4ff3..9719c57 100644 --- a/glsl/clear.frag.glsl +++ b/glsl/clear.frag.glsl @@ -3,7 +3,7 @@ precision highp float; precision highp int; -#define NUM_GBUFFERS 4 +#define NUM_GBUFFERS 2 void main() { for (int i = 0; i < NUM_GBUFFERS; i++) { diff --git a/glsl/copy.frag.glsl b/glsl/copy.frag.glsl index 0f5f8f7..4bd8c30 100644 --- a/glsl/copy.frag.glsl +++ b/glsl/copy.frag.glsl @@ -6,10 +6,30 @@ precision highp int; uniform sampler2D u_colmap; uniform sampler2D u_normap; +uniform float u_specExp; +uniform float u_remove; + varying vec3 v_position; varying vec3 v_normal; varying vec2 v_uv; +vec3 applyNormalMap(vec3 geomnor, vec3 normap) { + normap = normap * 2.0 - 1.0; + vec3 up = normalize(vec3(0.001, 1, 0.001)); + vec3 surftan = normalize(cross(geomnor, up)); + vec3 surfbinor = cross(geomnor, surftan); + return normap.y * surftan + normap.x * surfbinor + normap.z * geomnor; +} + +// http://aras-p.info/blog/2009/07/30/encoding-floats-to-rgba-the-final/ +// http://stackoverflow.com/questions/30242013/glsl-compressing-packing-multiple-0-1-colours-var4-into-a-single-var4-variab +float packRGBA( vec4 rgba ) { + return dot( floor(rgba.rgb*100.0), vec3(1000.0*1000.0, 1000.0, 1.0) ); +} + void main() { - // TODO: copy values into gl_FragData[0], [1], etc. + // Copy values into gl_FragData[0], [1], etc. + gl_FragData[0] = vec4(v_position, packRGBA(texture2D(u_colmap, v_uv))); + gl_FragData[1] = vec4(applyNormalMap(v_normal, texture2D(u_normap, v_uv).rgb), u_specExp); + //gl_FragData[2] = vec4(texture2D(u_colmap, v_uv).rgb, u_remove); } diff --git a/glsl/deferred/ambient.frag.glsl b/glsl/deferred/ambient.frag.glsl index 1fd4647..1326b6c 100644 --- a/glsl/deferred/ambient.frag.glsl +++ b/glsl/deferred/ambient.frag.glsl @@ -3,25 +3,40 @@ precision highp float; precision highp int; -#define NUM_GBUFFERS 4 +#define NUM_GBUFFERS 2 uniform sampler2D u_gbufs[NUM_GBUFFERS]; uniform sampler2D u_depth; +const vec4 SKY_COLOR = vec4(0.1, 0.14, 0.22, 0.1)*2.0; + varying vec2 v_uv; +// http://aras-p.info/blog/2009/07/30/encoding-floats-to-rgba-the-final/ +// http://stackoverflow.com/questions/30242013/glsl-compressing-packing-multiple-0-1-colours-var4-into-a-single-var4-variab +vec4 unpackRGBA( float v ) { + float r = floor(v/1000.0/1000.0); + v-=(r*1000.0*1000.0); + float g = floor(v/1000.0); + v-=(g*1000.0); + float b = floor(v); + return vec4(r,g,b,v)/100.0; +} + + void main() { vec4 gb0 = texture2D(u_gbufs[0], v_uv); vec4 gb1 = texture2D(u_gbufs[1], v_uv); - vec4 gb2 = texture2D(u_gbufs[2], v_uv); - vec4 gb3 = texture2D(u_gbufs[3], v_uv); + //vec4 gb2 = texture2D(u_gbufs[2], v_uv); + //vec4 gb3 = texture2D(u_gbufs[3], v_uv); float depth = texture2D(u_depth, v_uv).x; - // TODO: Extract needed properties from the g-buffers into local variables + + vec3 colmap = unpackRGBA(gb0.w).xyz; // The color map - unlit "albedo" (surface color) if (depth == 1.0) { gl_FragColor = vec4(0, 0, 0, 0); // set alpha to 0 return; } - gl_FragColor = vec4(0.1, 0.1, 0.1, 1); // TODO: replace this + gl_FragColor = SKY_COLOR*vec4(colmap, 1.0); } diff --git a/glsl/deferred/blinnphong-pointlight.frag.glsl b/glsl/deferred/blinnphong-pointlight.frag.glsl index b24a54a..885139b 100644 --- a/glsl/deferred/blinnphong-pointlight.frag.glsl +++ b/glsl/deferred/blinnphong-pointlight.frag.glsl @@ -2,14 +2,17 @@ precision highp float; precision highp int; -#define NUM_GBUFFERS 4 +#define NUM_GBUFFERS 2 uniform vec3 u_lightCol; uniform vec3 u_lightPos; uniform float u_lightRad; +uniform vec3 u_camPos; uniform sampler2D u_gbufs[NUM_GBUFFERS]; uniform sampler2D u_depth; +uniform int u_toon; + varying vec2 v_uv; vec3 applyNormalMap(vec3 geomnor, vec3 normap) { @@ -20,13 +23,31 @@ vec3 applyNormalMap(vec3 geomnor, vec3 normap) { return normap.y * surftan + normap.x * surfbinor + normap.z * geomnor; } +// http://aras-p.info/blog/2009/07/30/encoding-floats-to-rgba-the-final/ +// http://stackoverflow.com/questions/30242013/glsl-compressing-packing-multiple-0-1-colours-var4-into-a-single-var4-variab +vec4 unpackRGBA( float v ) { + float r = floor(v/1000.0/1000.0); + v-=(r*1000.0*1000.0); + float g = floor(v/1000.0); + v-=(g*1000.0); + float b = floor(v); + return vec4(r,g,b,v)/100.0; +} + void main() { vec4 gb0 = texture2D(u_gbufs[0], v_uv); vec4 gb1 = texture2D(u_gbufs[1], v_uv); - vec4 gb2 = texture2D(u_gbufs[2], v_uv); - vec4 gb3 = texture2D(u_gbufs[3], v_uv); + //vec4 gb2 = texture2D(u_gbufs[2], v_uv); + //vec4 gb3 = texture2D(u_gbufs[3], v_uv); float depth = texture2D(u_depth, v_uv).x; - // TODO: Extract needed properties from the g-buffers into local variables + // Extract needed properties from the g-buffers into local variables + vec3 pos = gb0.xyz; // World-space position + //vec3 geomnor = gb1.xyz; // Normals of the geometry as defined, without normal mapping + vec3 colmap = unpackRGBA(gb0.w).xyz; // The color map - unlit "albedo" (surface color) + vec3 nor = gb1.xyz; // The raw normal map (normals relative to the surface they're on) + //vec3 nor = applyNormalMap(geomnor, normap); // The true normals as we want to light them - with the normal map applied to the geometry normals (applyNormalMap above) + float specExp = gb1.w; + //float removeChannel = gb2.w; // If nothing was rendered to this pixel, set alpha to 0 so that the // postprocessing step can render the sky color. @@ -35,5 +56,35 @@ void main() { return; } - gl_FragColor = vec4(0, 0, 1, 1); // TODO: perform lighting calculations + float dist = length(pos-u_lightPos); + if (dist > u_lightRad){ + gl_FragColor = vec4(0, 0, 0, 0); + return; + } + + // Camera at (0,0,0) + vec3 V = normalize(u_camPos-pos); + vec3 L = normalize(u_lightPos-pos); + vec3 H = normalize(L+V); + + //float specExp = 10.0; + float diffIntense = max(dot(nor, L), 0.0); + float specIntense = pow(max(dot(nor, H), 0.0), specExp); + float falloff = (1.0-dist/u_lightRad)/pow(dist/u_lightRad, 0.7); +/* + if (removeChannel == 0.0) colmap.x = 0.0; + if (removeChannel == 1.0) colmap.y = 0.0; + if (removeChannel == 2.0) colmap.z = 0.0; +*/ + // Toon ramping + // Concept: http://prideout.net/blog/?p=22#toon + if (u_toon == 1){ + float steps = 4.0; + diffIntense = ceil(diffIntense*steps)/steps; + specIntense = ceil(specIntense*steps)/steps; + falloff = ceil(falloff*steps)/steps; + gl_FragColor = vec4(falloff*(diffIntense*colmap*u_lightCol+specIntense*vec3(1.0)), falloff); + } else { + gl_FragColor = vec4(falloff*(diffIntense*colmap*u_lightCol+specIntense*vec3(1.0)), falloff); + } } diff --git a/glsl/deferred/debug.frag.glsl b/glsl/deferred/debug.frag.glsl index 9cbfae4..cf569e4 100644 --- a/glsl/deferred/debug.frag.glsl +++ b/glsl/deferred/debug.frag.glsl @@ -2,7 +2,7 @@ precision highp float; precision highp int; -#define NUM_GBUFFERS 4 +#define NUM_GBUFFERS 2 uniform int u_debug; uniform sampler2D u_gbufs[NUM_GBUFFERS]; @@ -20,33 +20,46 @@ vec3 applyNormalMap(vec3 geomnor, vec3 normap) { return normap.y * surftan + normap.x * surfbinor + normap.z * geomnor; } +// http://aras-p.info/blog/2009/07/30/encoding-floats-to-rgba-the-final/ +// http://stackoverflow.com/questions/30242013/glsl-compressing-packing-multiple-0-1-colours-var4-into-a-single-var4-variab +vec4 unpackRGBA( float v ) { + float r = floor(v/1000.0/1000.0); + v-=(r*1000.0*1000.0); + float g = floor(v/1000.0); + v-=(g*1000.0); + float b = floor(v); + return vec4(r,g,b,v)/100.0; +} + void main() { vec4 gb0 = texture2D(u_gbufs[0], v_uv); vec4 gb1 = texture2D(u_gbufs[1], v_uv); - vec4 gb2 = texture2D(u_gbufs[2], v_uv); - vec4 gb3 = texture2D(u_gbufs[3], v_uv); + //vec4 gb2 = texture2D(u_gbufs[2], v_uv); + //vec4 gb3 = texture2D(u_gbufs[3], v_uv); float depth = texture2D(u_depth, v_uv).x; - // TODO: Extract needed properties from the g-buffers into local variables + // Extract needed properties from the g-buffers into local variables // These definitions are suggested for starting out, but you will probably want to change them. - vec3 pos; // World-space position - vec3 geomnor; // Normals of the geometry as defined, without normal mapping - vec3 colmap; // The color map - unlit "albedo" (surface color) - vec3 normap; // The raw normal map (normals relative to the surface they're on) - vec3 nor; // The true normals as we want to light them - with the normal map applied to the geometry normals (applyNormalMap above) + vec3 pos = gb0.xyz; // World-space position + //vec3 geomnor = gb1.xyz; // Normals of the geometry as defined, without normal mapping + vec3 colmap = unpackRGBA(gb0.w).xyz; // The color map - unlit "albedo" (surface color) + //vec3 normap = gb3.xyz; // The raw normal map (normals relative to the surface they're on) + //vec3 nor = applyNormalMap(geomnor, normap); // The true normals as we want to light them - with the normal map applied to the geometry normals (applyNormalMap above) + vec3 nor = gb1.xyz; if (u_debug == 0) { gl_FragColor = vec4(vec3(depth), 1.0); } else if (u_debug == 1) { gl_FragColor = vec4(abs(pos) * 0.1, 1.0); } else if (u_debug == 2) { - gl_FragColor = vec4(abs(geomnor), 1.0); + //gl_FragColor = vec4(abs(geomnor), 1.0); + gl_FragColor = vec4(abs(nor), 1.0); } else if (u_debug == 3) { gl_FragColor = vec4(colmap, 1.0); - } else if (u_debug == 4) { + } /*else if (u_debug == 4) { gl_FragColor = vec4(normap, 1.0); } else if (u_debug == 5) { gl_FragColor = vec4(abs(nor), 1.0); - } else { + }*/ else { gl_FragColor = vec4(1, 0, 1, 1); } } diff --git a/glsl/post/direct.frag.glsl b/glsl/post/direct.frag.glsl new file mode 100644 index 0000000..dda4a2d --- /dev/null +++ b/glsl/post/direct.frag.glsl @@ -0,0 +1,20 @@ +#version 100 +precision highp float; +precision highp int; + +uniform sampler2D u_color; +uniform vec2 u_screen_inv; + +varying vec2 v_uv; + +const vec4 SKY_COLOR = vec4(0.01, 0.14, 0.42, 1.0); + +void main() { + vec4 color = texture2D(u_color, v_uv); + + if (color.a == 0.0) { + color = SKY_COLOR; + } + + gl_FragColor = color; +} diff --git a/glsl/post/one.frag.glsl b/glsl/post/one.frag.glsl index 94191cd..643be0c 100644 --- a/glsl/post/one.frag.glsl +++ b/glsl/post/one.frag.glsl @@ -3,18 +3,32 @@ precision highp float; precision highp int; uniform sampler2D u_color; +uniform vec2 u_screen_inv; varying vec2 v_uv; const vec4 SKY_COLOR = vec4(0.01, 0.14, 0.42, 1.0); +// Gaussian +const vec3 G = vec3(0.399, 0.242, 0.054); + void main() { vec4 color = texture2D(u_color, v_uv); if (color.a == 0.0) { - gl_FragColor = SKY_COLOR; - return; + color = SKY_COLOR; } - gl_FragColor = color; + vec4 L2 = texture2D(u_color,vec2(v_uv.x-u_screen_inv.x*2.0,v_uv.y)); + vec4 L1 = texture2D(u_color,vec2(v_uv.x-u_screen_inv.x*1.0,v_uv.y)); + vec4 R1 = texture2D(u_color,vec2(v_uv.x+u_screen_inv.x*1.0,v_uv.y)); + vec4 R2 = texture2D(u_color,vec2(v_uv.x+u_screen_inv.x*2.0,v_uv.y)); + + color = color*color.a; + L2 = L2*L2.a; + L1 = L1*L1.a; + R1 = R1*R1.a; + R2 = R2*R2.a; + + gl_FragColor = L2*G.z+L1*G.y+color*G.x+R1*G.y+R2*G.z; } diff --git a/glsl/post/toon1.frag.glsl b/glsl/post/toon1.frag.glsl new file mode 100644 index 0000000..e96daae --- /dev/null +++ b/glsl/post/toon1.frag.glsl @@ -0,0 +1,38 @@ +#version 100 +precision highp float; +precision highp int; + +uniform sampler2D u_color; +uniform vec2 u_screen_inv; + +varying vec2 v_uv; + +const vec4 SKY_COLOR = vec4(0.01, 0.14, 0.42, 1.0); + +void main() { + vec4 color = texture2D(u_color, v_uv); + + // https://en.wikipedia.org/wiki/Sobel_operator + + vec4 L1 = texture2D(u_color,vec2( + v_uv.x-u_screen_inv.x, + v_uv.y)); + vec4 L1T = texture2D(u_color,vec2( + v_uv.x-u_screen_inv.x, + v_uv.y-u_screen_inv.y)); + vec4 L1B = texture2D(u_color,vec2( + v_uv.x-u_screen_inv.x, + v_uv.y+u_screen_inv.y)); + + vec4 R1 = texture2D(u_color,vec2( + v_uv.x+u_screen_inv.x, + v_uv.y)); + vec4 R1T = texture2D(u_color,vec2( + v_uv.x+u_screen_inv.x, + v_uv.y-u_screen_inv.y)); + vec4 R1B = texture2D(u_color,vec2( + v_uv.x+u_screen_inv.x, + v_uv.y+u_screen_inv.y)); + + gl_FragColor = -L1*1.0-L1T*1.0-L1B*1.0+R1*1.0+R1T*1.0+R1B*1.0; +} diff --git a/glsl/post/toon2.frag.glsl b/glsl/post/toon2.frag.glsl new file mode 100644 index 0000000..370392d --- /dev/null +++ b/glsl/post/toon2.frag.glsl @@ -0,0 +1,49 @@ +#version 100 +precision highp float; +precision highp int; + +uniform sampler2D u_color; +uniform sampler2D o_color; +uniform vec2 u_screen_inv; + +varying vec2 v_uv; + +void main() { + vec4 color = texture2D(u_color, v_uv); + vec4 color_o = texture2D(o_color, v_uv); + + vec4 T1 = texture2D(u_color,vec2( + v_uv.x, + v_uv.y-u_screen_inv.y)); + vec4 T1L = texture2D(u_color,vec2( + v_uv.x-u_screen_inv.x, + v_uv.y-u_screen_inv.y)); + vec4 T1R = texture2D(u_color,vec2( + v_uv.x+u_screen_inv.x, + v_uv.y-u_screen_inv.y)); + + vec4 B1 = texture2D(u_color,vec2( + v_uv.x, + v_uv.y+u_screen_inv.y)); + vec4 B1L = texture2D(u_color,vec2( + v_uv.x-u_screen_inv.x, + v_uv.y+u_screen_inv.y)); + vec4 B1R = texture2D(u_color,vec2( + v_uv.x+u_screen_inv.x, + v_uv.y+u_screen_inv.y)); + + vec4 blend = -T1*1.0-T1L*1.0-T1R*1.0+B1*1.0+B1L*1.0+B1R*1.0; + + // Edge ramping + blend = vec4(vec3(max(max(blend.x, blend.y), blend.z)), 1); + + if (blend.x <= 0.3){ + blend = vec4(vec3(0.0), 1); + } else { + blend = vec4(1.0); + } + + blend = (1.0-blend) * color_o; + + gl_FragColor = blend; +} diff --git a/glsl/post/two.frag.glsl b/glsl/post/two.frag.glsl new file mode 100644 index 0000000..0891068 --- /dev/null +++ b/glsl/post/two.frag.glsl @@ -0,0 +1,39 @@ +#version 100 +precision highp float; +precision highp int; + +uniform sampler2D u_color; +uniform sampler2D o_color; +uniform vec2 u_screen_inv; + +varying vec2 v_uv; + +const vec4 SKY_COLOR = vec4(0.01, 0.14, 0.42, 1.0); + +// Gaussian +const vec3 G = vec3(0.399, 0.242, 0.054); + +void main() { + vec4 color = texture2D(u_color, v_uv); + vec4 color_o = texture2D(o_color, v_uv); + + if (color.a == 0.0) { + color = SKY_COLOR; + } + + vec4 T2 = texture2D(u_color,vec2(v_uv.x, v_uv.y-u_screen_inv.y*2.0)); + vec4 T1 = texture2D(u_color,vec2(v_uv.x, v_uv.y-u_screen_inv.y*1.0)); + vec4 B1 = texture2D(u_color,vec2(v_uv.x, v_uv.y+u_screen_inv.y*1.0)); + vec4 B2 = texture2D(u_color,vec2(v_uv.x, v_uv.y+u_screen_inv.y*2.0)); + + // Add to original + vec4 blend = color_o + T2*G.z+T1*G.y+color*G.x+B1*G.y+B2*G.z; + + // Tone mapping + blend.x = 0.7*pow(blend.x, 1.0); + blend.y = 0.7*pow(blend.y, 1.0); + blend.z = 0.7*pow(blend.z, 1.0); + blend.a = 0.7*pow(blend.a, 1.0); + + gl_FragColor = blend; +} diff --git a/glsl/red.frag.glsl b/glsl/red.frag.glsl index f8ef1ec..e41993b 100644 --- a/glsl/red.frag.glsl +++ b/glsl/red.frag.glsl @@ -3,5 +3,5 @@ precision highp float; precision highp int; void main() { - gl_FragColor = vec4(1, 0, 0, 1); + gl_FragColor = vec4(1, 0, 0, 0.1); } diff --git a/img/aabb-compare.png b/img/aabb-compare.png new file mode 100644 index 0000000..07e5096 Binary files /dev/null and b/img/aabb-compare.png differ diff --git a/img/bloom-shading.png b/img/bloom-shading.png new file mode 100644 index 0000000..5eb6a1c Binary files /dev/null and b/img/bloom-shading.png differ diff --git a/img/color-packed.png b/img/color-packed.png new file mode 100644 index 0000000..2df402f Binary files /dev/null and b/img/color-packed.png differ diff --git a/img/debugs.png b/img/debugs.png new file mode 100644 index 0000000..c602616 Binary files /dev/null and b/img/debugs.png differ diff --git a/img/default-aabb-debug.png b/img/default-aabb-debug.png new file mode 100644 index 0000000..e7d853d Binary files /dev/null and b/img/default-aabb-debug.png differ diff --git a/img/default-aabb-render.png b/img/default-aabb-render.png new file mode 100644 index 0000000..f858a4e Binary files /dev/null and b/img/default-aabb-render.png differ diff --git a/img/deferred-render.png b/img/deferred-render.png new file mode 100644 index 0000000..5a2389b Binary files /dev/null and b/img/deferred-render.png differ diff --git a/img/depth.png b/img/depth.png new file mode 100644 index 0000000..6db4d70 Binary files /dev/null and b/img/depth.png differ diff --git a/img/fragment-position.png b/img/fragment-position.png new file mode 100644 index 0000000..6787fb9 Binary files /dev/null and b/img/fragment-position.png differ diff --git a/img/gbuf-opt.png b/img/gbuf-opt.png new file mode 100644 index 0000000..60d1f21 Binary files /dev/null and b/img/gbuf-opt.png differ diff --git a/img/geometry-normal.png b/img/geometry-normal.png new file mode 100644 index 0000000..189d0d3 Binary files /dev/null and b/img/geometry-normal.png differ diff --git a/img/improved-aabb-debug.png b/img/improved-aabb-debug.png new file mode 100644 index 0000000..262096d Binary files /dev/null and b/img/improved-aabb-debug.png differ diff --git a/img/improved-aabb-render.png b/img/improved-aabb-render.png new file mode 100644 index 0000000..daa309b Binary files /dev/null and b/img/improved-aabb-render.png differ diff --git a/img/light-time.png b/img/light-time.png new file mode 100644 index 0000000..365cc5a Binary files /dev/null and b/img/light-time.png differ diff --git a/img/packing-color.png b/img/packing-color.png new file mode 100644 index 0000000..cb8a3a2 Binary files /dev/null and b/img/packing-color.png differ diff --git a/img/packing-comparison.png b/img/packing-comparison.png new file mode 100644 index 0000000..4cb2022 Binary files /dev/null and b/img/packing-comparison.png differ diff --git a/img/packing-demo.png b/img/packing-demo.png new file mode 100644 index 0000000..b65372b Binary files /dev/null and b/img/packing-demo.png differ diff --git a/img/packing-render.png b/img/packing-render.png new file mode 100644 index 0000000..7840b8c Binary files /dev/null and b/img/packing-render.png differ diff --git a/img/scissor-effect.png b/img/scissor-effect.png new file mode 100644 index 0000000..f358341 Binary files /dev/null and b/img/scissor-effect.png differ diff --git a/img/scissor-pure.png b/img/scissor-pure.png new file mode 100644 index 0000000..bd854ea Binary files /dev/null and b/img/scissor-pure.png differ diff --git a/img/spec-exp.png b/img/spec-exp.png new file mode 100644 index 0000000..4281c0c Binary files /dev/null and b/img/spec-exp.png differ diff --git a/img/surface-normal.png b/img/surface-normal.png new file mode 100644 index 0000000..94ba053 Binary files /dev/null and b/img/surface-normal.png differ diff --git a/img/texture-color.png b/img/texture-color.png new file mode 100644 index 0000000..64e07c5 Binary files /dev/null and b/img/texture-color.png differ diff --git a/img/texture-normal.png b/img/texture-normal.png new file mode 100644 index 0000000..f6e9ec5 Binary files /dev/null and b/img/texture-normal.png differ diff --git a/img/thumb.png b/img/thumb.png index 9ec8ed0..1e3aabe 100644 Binary files a/img/thumb.png and b/img/thumb.png differ diff --git a/img/toon-shading.png b/img/toon-shading.png new file mode 100644 index 0000000..9bda094 Binary files /dev/null and b/img/toon-shading.png differ diff --git a/img/video.png b/img/video.png index 9ec8ed0..531a926 100644 Binary files a/img/video.png and b/img/video.png differ diff --git a/js/deferredRender.js b/js/deferredRender.js index b1f238b..f9c3548 100644 --- a/js/deferredRender.js +++ b/js/deferredRender.js @@ -27,13 +27,13 @@ // CHECKITOUT: START HERE! You can even uncomment this: //debugger; - + /* { // TODO: this block should be removed after testing renderFullScreenQuad gl.bindFramebuffer(gl.FRAMEBUFFER, null); - // TODO: Implement/test renderFullScreenQuad first renderFullScreenQuad(R.progRed); return; } + */ R.pass_copy.render(state); @@ -41,13 +41,20 @@ // Do a debug render instead of a regular render // Don't do any post-processing in debug mode R.pass_debug.render(state); + } else if (cfg && cfg.debugScissor){ + R.pass_deferred.renderScissor(state); } else { // * Deferred pass and postprocessing pass(es) - // TODO: uncomment these - //R.pass_deferred.render(state); - //R.pass_post1.render(state); - - // OPTIONAL TODO: call more postprocessing passes, if any + R.pass_deferred.render(state); + if (cfg.effects == -1){ + R.pass_post1.directRender(state); + } else if (cfg.effects == 0){ + R.pass_post1.render(state); + R.pass_post2.render(state); + } else if (cfg.effects == 1){ + R.pass_postT1.render(state); + R.pass_postT2.render(state); + } } }; @@ -56,22 +63,20 @@ */ R.pass_copy.render = function(state) { // * Bind the framebuffer R.pass_copy.fbo - // TODO: ^ + gl.bindFramebuffer(gl.FRAMEBUFFER, R.pass_copy.fbo); // * Clear screen using R.progClear - TODO: renderFullScreenQuad(R.progClear); + renderFullScreenQuad(R.progClear); // * Clear depth buffer to value 1.0 using gl.clearDepth and gl.clear - // TODO: ^ - // TODO: ^ + gl.clearDepth(1.0); + gl.clear(gl.DEPTH_BUFFER_BIT); // * "Use" the program R.progCopy.prog - // TODO: ^ - // TODO: Write glsl/copy.frag.glsl + gl.useProgram(R.progCopy.prog); - var m = state.cameraMat.elements; // * Upload the camera matrix m to the uniform R.progCopy.u_cameraMat // using gl.uniformMatrix4fv - // TODO: ^ + gl.uniformMatrix4fv(R.progCopy.u_cameraMat, false, state.cameraMat.elements); // * Draw the scene drawScene(state); @@ -79,13 +84,12 @@ var drawScene = function(state) { for (var i = 0; i < state.models.length; i++) { - var m = state.models[i]; // If you want to render one model many times, note: // readyModelForDraw only needs to be called once. - readyModelForDraw(R.progCopy, m); - - drawReadyModel(m); + readyModelForDraw(R.progCopy, state.models[i]); + + drawReadyModel(state.models[i]); } }; @@ -102,12 +106,65 @@ renderFullScreenQuad(R.prog_Debug); }; + /** + * 'deferred' pass: Scissor test debug view + */ + R.pass_deferred.renderScissor = function(state) { + // * Bind R.pass_deferred.fbo to write into for later postprocessing + //gl.bindFramebuffer(gl.FRAMEBUFFER, R.pass_deferred.fbo); + gl.bindFramebuffer(gl.FRAMEBUFFER, null); + + // * Clear depth to 1.0 and color to black + gl.clearColor(0.0, 0.0, 0.0, 0.0); + gl.clearDepth(1.0); + gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT); + + // * _ADD_ together the result of each lighting pass + + // Enable blending and use gl.blendFunc to blend with: + // color = 1 * src_color + 1 * dst_color + gl.blendFunc(gl.SRC_ALPHA, gl.DST_ALPHA); + gl.enable(gl.BLEND); + + // A loop here, over the values in R.lights, which sets the + // uniforms R.prog_BlinnPhong_PointLight.u_lightPos/Col/Rad etc., + // then does renderFullScreenQuad(R.prog_BlinnPhong_PointLight). + + // In the lighting loop, use the scissor test optimization + // Enable gl.SCISSOR_TEST, render all lights, then disable it. + // + // getScissorForLight returns null if the scissor is off the screen. + // Otherwise, it returns an array [xmin, ymin, width, height]. + // + // var sc = getScissorForLight(state.viewMat, state.projMat, light); + gl.enable(gl.SCISSOR_TEST); + + for (var i = R.lights.length - 1; i >= 0; i--) { + var L = R.lights[i]; + var sc; + if (cfg.improvedAABB){ + var sc = getScissorForLightI(state.viewMat, state.projMat, L); + } else { + var sc = getScissorForLight(state.viewMat, state.projMat, L); + } + if (sc != null){ + gl.scissor(sc[0], sc[1], sc[2], sc[3]); + renderFullScreenQuad(R.progRed); + } + }; + + gl.disable(gl.SCISSOR_TEST); + // Disable blending so that it doesn't affect other code + gl.disable(gl.BLEND); + }; + /** * 'deferred' pass: Add lighting results for each individual light */ R.pass_deferred.render = function(state) { // * Bind R.pass_deferred.fbo to write into for later postprocessing gl.bindFramebuffer(gl.FRAMEBUFFER, R.pass_deferred.fbo); + //gl.bindFramebuffer(gl.FRAMEBUFFER, null); // * Clear depth to 1.0 and color to black gl.clearColor(0.0, 0.0, 0.0, 0.0); @@ -115,10 +172,11 @@ gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT); // * _ADD_ together the result of each lighting pass - + // Enable blending and use gl.blendFunc to blend with: // color = 1 * src_color + 1 * dst_color - // TODO: ^ + gl.blendFunc(gl.ONE, gl.ONE); + gl.enable(gl.BLEND); // * Bind/setup the ambient pass, and render using fullscreen quad bindTexturesForLightPass(R.prog_Ambient); @@ -127,11 +185,11 @@ // * Bind/setup the Blinn-Phong pass, and render using fullscreen quad bindTexturesForLightPass(R.prog_BlinnPhong_PointLight); - // TODO: add a loop here, over the values in R.lights, which sets the + // A loop here, over the values in R.lights, which sets the // uniforms R.prog_BlinnPhong_PointLight.u_lightPos/Col/Rad etc., // then does renderFullScreenQuad(R.prog_BlinnPhong_PointLight). - // TODO: In the lighting loop, use the scissor test optimization + // In the lighting loop, use the scissor test optimization // Enable gl.SCISSOR_TEST, render all lights, then disable it. // // getScissorForLight returns null if the scissor is off the screen. @@ -139,6 +197,41 @@ // // var sc = getScissorForLight(state.viewMat, state.projMat, light); + gl.uniform1i(R.prog_BlinnPhong_PointLight.u_toon, cfg.effects); + gl.uniform3f(R.prog_BlinnPhong_PointLight.u_camPos, state.cameraPos[0], state.cameraPos[1], state.cameraPos[2]); + + if (cfg.scissoring){ + gl.enable(gl.SCISSOR_TEST); + + for (var i = R.lights.length - 1; i >= 0; i--) { + var L = R.lights[i]; + var sc; + if (cfg.improvedAABB){ + var sc = getScissorForLightI(state.viewMat, state.projMat, L); + } else { + var sc = getScissorForLight(state.viewMat, state.projMat, L); + } + if (sc != null){ + gl.scissor(sc[0], sc[1], sc[2], sc[3]); + + gl.uniform3f(R.prog_BlinnPhong_PointLight.u_lightPos, L.pos[0], L.pos[1], L.pos[2]); + gl.uniform3f(R.prog_BlinnPhong_PointLight.u_lightCol, L.col[0], L.col[1], L.col[2]); + gl.uniform1f(R.prog_BlinnPhong_PointLight.u_lightRad, L.rad); + renderFullScreenQuad(R.prog_BlinnPhong_PointLight); + } + }; + + gl.disable(gl.SCISSOR_TEST); + } else { + for (var i = R.lights.length - 1; i >= 0; i--) { + var L = R.lights[i]; + + gl.uniform3f(R.prog_BlinnPhong_PointLight.u_lightPos, L.pos[0], L.pos[1], L.pos[2]); + gl.uniform3f(R.prog_BlinnPhong_PointLight.u_lightCol, L.col[0], L.col[1], L.col[2]); + gl.uniform1f(R.prog_BlinnPhong_PointLight.u_lightRad, L.rad); + renderFullScreenQuad(R.prog_BlinnPhong_PointLight); + }; + } // Disable blending so that it doesn't affect other code gl.disable(gl.BLEND); }; @@ -158,12 +251,37 @@ gl.uniform1i(prog.u_depth, R.NUM_GBUFFERS); }; + /** + * 'post0' pass: Direct render + */ + R.pass_post1.directRender = function(state) { + // * Unbind any existing framebuffer (if there are no more passes) + gl.bindFramebuffer(gl.FRAMEBUFFER, null); + + // * Clear the framebuffer depth to 1.0 + gl.clearDepth(1.0); + gl.clear(gl.DEPTH_BUFFER_BIT); + + // * Bind the postprocessing shader program + gl.useProgram(R.progPostDirect.prog); + + gl.activeTexture(gl.TEXTURE0); + gl.bindTexture(gl.TEXTURE_2D, R.pass_deferred.colorTex); + gl.uniform1i(R.progPostDirect.u_color, 0); + + renderFullScreenQuad(R.progPostDirect); + }; + + /******************************************************************************** + * Bloom shading + ********************************************************************************/ + /** * 'post1' pass: Perform (first) pass of post-processing */ R.pass_post1.render = function(state) { // * Unbind any existing framebuffer (if there are no more passes) - gl.bindFramebuffer(gl.FRAMEBUFFER, null); + gl.bindFramebuffer(gl.FRAMEBUFFER, R.pass_post1.fbo); // * Clear the framebuffer depth to 1.0 gl.clearDepth(1.0); @@ -174,16 +292,122 @@ // * Bind the deferred pass's color output as a texture input // Set gl.TEXTURE0 as the gl.activeTexture unit - // TODO: ^ + gl.activeTexture(gl.TEXTURE0); // Bind the TEXTURE_2D, R.pass_deferred.colorTex to the active texture unit - // TODO: ^ + gl.bindTexture(gl.TEXTURE_2D, R.pass_deferred.colorTex); // Configure the R.progPost1.u_color uniform to point at texture unit 0 gl.uniform1i(R.progPost1.u_color, 0); + gl.uniform2f(R.progPost1.u_screen_inv, 1.0/state.screenDim.w, 1.0/state.screenDim.h); + // * Render a fullscreen quad to perform shading on renderFullScreenQuad(R.progPost1); }; + /** + * 'post2' pass: Perform pass of post-processing + */ + R.pass_post2.render = function(state) { + // * Unbind any existing framebuffer (if there are no more passes) + gl.bindFramebuffer(gl.FRAMEBUFFER, null); + + // * Clear the framebuffer depth to 1.0 + gl.clearDepth(1.0); + gl.clear(gl.DEPTH_BUFFER_BIT); + + // * Bind the postprocessing shader program + gl.useProgram(R.progPost2.prog); + + // * Bind the deferred pass's color output as a texture input + // Set gl.TEXTURE0 as the gl.activeTexture unit + gl.activeTexture(gl.TEXTURE0); + // Bind the TEXTURE_2D, R.pass_deferred.colorTex to the active texture unit + gl.bindTexture(gl.TEXTURE_2D, R.pass_deferred.colorTex); + // Configure the R.progPost2.u_color uniform to point at texture unit 0 + gl.uniform1i(R.progPost2.o_color, 0); + + // * Bind the deferred pass's color output as a texture input + // Set gl.TEXTURE0 as the gl.activeTexture unit + gl.activeTexture(gl.TEXTURE1); + // Bind the TEXTURE_2D, R.pass_deferred.colorTex to the active texture unit + gl.bindTexture(gl.TEXTURE_2D, R.pass_post1.colorTex); + // Configure the R.progPost2.u_color uniform to point at texture unit 0 + gl.uniform1i(R.progPost2.u_color, 1); + + gl.uniform2f(R.progPost2.u_screen_inv, 1.0/state.screenDim.w, 1.0/state.screenDim.h); + + // * Render a fullscreen quad to perform shading on + renderFullScreenQuad(R.progPost2); + }; + + /******************************************************************************** + * Toon shading edge detector + ********************************************************************************/ + + /** + * 'post1' pass: Perform (first) pass of post-processing + */ + R.pass_postT1.render = function(state) { + // * Unbind any existing framebuffer (if there are no more passes) + gl.bindFramebuffer(gl.FRAMEBUFFER, R.pass_postT1.fbo); + + // * Clear the framebuffer depth to 1.0 + gl.clearDepth(1.0); + gl.clear(gl.DEPTH_BUFFER_BIT); + + // * Bind the postprocessing shader program + gl.useProgram(R.progPostToon1.prog); + + // * Bind the deferred pass's color output as a texture input + // Set gl.TEXTURE0 as the gl.activeTexture unit + gl.activeTexture(gl.TEXTURE0); + // Bind the TEXTURE_2D, R.pass_deferred.colorTex to the active texture unit + gl.bindTexture(gl.TEXTURE_2D, R.pass_deferred.colorTex); + // Configure the R.progPostToon1.u_color uniform to point at texture unit 0 + gl.uniform1i(R.progPostToon1.u_color, 0); + + gl.uniform2f(R.progPostToon1.u_screen_inv, 1.0/state.screenDim.w, 1.0/state.screenDim.h); + + // * Render a fullscreen quad to perform shading on + renderFullScreenQuad(R.progPostToon1); + }; + + /** + * 'post2' pass: Perform pass of post-processing + */ + R.pass_postT2.render = function(state) { + // * Unbind any existing framebuffer (if there are no more passes) + gl.bindFramebuffer(gl.FRAMEBUFFER, null); + + // * Clear the framebuffer depth to 1.0 + gl.clearDepth(1.0); + gl.clear(gl.DEPTH_BUFFER_BIT); + + // * Bind the postprocessing shader program + gl.useProgram(R.progPostToon2.prog); + + // * Bind the deferred pass's color output as a texture input + // Set gl.TEXTURE0 as the gl.activeTexture unit + gl.activeTexture(gl.TEXTURE0); + // Bind the TEXTURE_2D, R.pass_deferred.colorTex to the active texture unit + gl.bindTexture(gl.TEXTURE_2D, R.pass_deferred.colorTex); + // Configure the R.progPostToon2.u_color uniform to point at texture unit 0 + gl.uniform1i(R.progPostToon2.o_color, 0); + + // * Bind the deferred pass's color output as a texture input + // Set gl.TEXTURE0 as the gl.activeTexture unit + gl.activeTexture(gl.TEXTURE1); + // Bind the TEXTURE_2D, R.pass_deferred.colorTex to the active texture unit + gl.bindTexture(gl.TEXTURE_2D, R.pass_postT1.colorTex); + // Configure the R.progPostToon2.u_color uniform to point at texture unit 0 + gl.uniform1i(R.progPostToon2.u_color, 1); + + gl.uniform2f(R.progPostToon2.u_screen_inv, 1.0/state.screenDim.w, 1.0/state.screenDim.h); + + // * Render a fullscreen quad to perform shading on + renderFullScreenQuad(R.progPostToon2); + }; + var renderFullScreenQuad = (function() { // The variables in this function are private to the implementation of // renderFullScreenQuad. They work like static local variables in C++. @@ -204,13 +428,13 @@ var init = function() { // Create a new buffer with gl.createBuffer, and save it as vbo. - // TODO: ^ + vbo = gl.createBuffer(); // Bind the VBO as the gl.ARRAY_BUFFER - // TODO: ^ + gl.bindBuffer(gl.ARRAY_BUFFER, vbo); // Upload the positions array to the currently-bound array buffer // using gl.bufferData in static draw mode. - // TODO: ^ + gl.bufferData(gl.ARRAY_BUFFER, positions, gl.STATIC_DRAW); }; return function(prog) { @@ -223,16 +447,16 @@ gl.useProgram(prog.prog); // Bind the VBO as the gl.ARRAY_BUFFER - // TODO: ^ + gl.bindBuffer(gl.ARRAY_BUFFER, vbo); // Enable the bound buffer as the vertex attrib array for // prog.a_position, using gl.enableVertexAttribArray - // TODO: ^ + gl.enableVertexAttribArray(prog.a_position); // Use gl.vertexAttribPointer to tell WebGL the type/layout for // prog.a_position's access pattern. - // TODO: ^ + gl.vertexAttribPointer(prog.a_position, 3, gl.FLOAT, false, 0, 0); // Use gl.drawArrays (or gl.drawElements) to draw your quad. - // TODO: ^ + gl.drawArrays(gl.TRIANGLE_STRIP, 0, 4); // Unbind the array buffer. gl.bindBuffer(gl.ARRAY_BUFFER, null); diff --git a/js/deferredSetup.js b/js/deferredSetup.js index 115b016..171d29c 100644 --- a/js/deferredSetup.js +++ b/js/deferredSetup.js @@ -6,9 +6,12 @@ R.pass_debug = {}; R.pass_deferred = {}; R.pass_post1 = {}; + R.pass_post2 = {}; + R.pass_postT1 = {}; + R.pass_postT2 = {}; R.lights = []; - R.NUM_GBUFFERS = 4; + R.NUM_GBUFFERS = 2; /** * Set up the deferred pipeline framebuffer objects and textures. @@ -18,6 +21,8 @@ loadAllShaderPrograms(); R.pass_copy.setup(); R.pass_deferred.setup(); + R.pass_post1.setup(); + R.pass_postT1.setup(); }; // TODO: Edit if you want to change the light initial positions @@ -94,6 +99,40 @@ gl_draw_buffers.drawBuffersWEBGL([gl_draw_buffers.COLOR_ATTACHMENT0_WEBGL]); }; + /** + * Bloom shading: Create/configure framebuffer between "poat1" and "post2" stages + */ + R.pass_post1.setup = function() { + // * Create the FBO + R.pass_post1.fbo = gl.createFramebuffer(); + // * Create, bind, and store a single color target texture for the FBO + R.pass_post1.colorTex = createAndBindColorTargetTexture( + R.pass_post1.fbo, gl_draw_buffers.COLOR_ATTACHMENT0_WEBGL); + + // * Check for framebuffer errors + abortIfFramebufferIncomplete(R.pass_post1.fbo); + // * Tell the WEBGL_draw_buffers extension which FBO attachments are + // being used. (This extension allows for multiple render targets.) + gl_draw_buffers.drawBuffersWEBGL([gl_draw_buffers.COLOR_ATTACHMENT0_WEBGL]); + }; + + /** + * Toon shading: Create/configure framebuffer between "deferred" and "post1" stages + */ + R.pass_postT1.setup = function() { + // * Create the FBO + R.pass_postT1.fbo = gl.createFramebuffer(); + // * Create, bind, and store a single color target texture for the FBO + R.pass_postT1.colorTex = createAndBindColorTargetTexture( + R.pass_postT1.fbo, gl_draw_buffers.COLOR_ATTACHMENT0_WEBGL); + + // * Check for framebuffer errors + abortIfFramebufferIncomplete(R.pass_postT1.fbo); + // * Tell the WEBGL_draw_buffers extension which FBO attachments are + // being used. (This extension allows for multiple render targets.) + gl_draw_buffers.drawBuffersWEBGL([gl_draw_buffers.COLOR_ATTACHMENT0_WEBGL]); + }; + /** * Loads all of the shader programs used in the pipeline. */ @@ -111,6 +150,9 @@ p.a_normal = gl.getAttribLocation(prog, 'a_normal'); p.a_uv = gl.getAttribLocation(prog, 'a_uv'); + p.u_specExp = gl.getUniformLocation(prog, 'u_specExp'); + p.u_remove = gl.getUniformLocation(prog, 'u_remove'); + // Save the object into this variable for access later R.progCopy = p; }); @@ -137,6 +179,7 @@ p.u_lightPos = gl.getUniformLocation(p.prog, 'u_lightPos'); p.u_lightCol = gl.getUniformLocation(p.prog, 'u_lightCol'); p.u_lightRad = gl.getUniformLocation(p.prog, 'u_lightRad'); + p.u_toon = gl.getUniformLocation(p.prog, 'u_toon'); R.prog_BlinnPhong_PointLight = p; }); @@ -146,12 +189,43 @@ R.prog_Debug = p; }); + loadPostProgram('direct', function(p) { + p.u_color = gl.getUniformLocation(p.prog, 'u_color'); + // Save the object into this variable for access later + R.progPostDirect = p; + }); + loadPostProgram('one', function(p) { p.u_color = gl.getUniformLocation(p.prog, 'u_color'); + p.u_screen_inv = gl.getUniformLocation(p.prog, 'u_screen_inv'); // Save the object into this variable for access later R.progPost1 = p; }); + loadPostProgram('two', function(p) { + p.o_color = gl.getUniformLocation(p.prog, 'o_color'); + p.u_color = gl.getUniformLocation(p.prog, 'u_color'); + p.u_screen_inv = gl.getUniformLocation(p.prog, 'u_screen_inv'); + // Save the object into this variable for access later + R.progPost2 = p; + }); + + loadPostProgram('toon1', function(p) { + p.u_color = gl.getUniformLocation(p.prog, 'u_color'); + p.u_screen_inv = gl.getUniformLocation(p.prog, 'u_screen_inv'); + // Save the object into this variable for access later + R.progPostToon1 = p; + }); + + loadPostProgram('toon2', function(p) { + p.o_color = gl.getUniformLocation(p.prog, 'o_color'); + p.u_color = gl.getUniformLocation(p.prog, 'u_color'); + p.u_screen_inv = gl.getUniformLocation(p.prog, 'u_screen_inv'); + // Save the object into this variable for access later + R.progPostToon2 = p; + }); + + // TODO: If you add more passes, load and set up their shader programs. }; diff --git a/js/framework.js b/js/framework.js index a0653a5..4a49a52 100644 --- a/js/framework.js +++ b/js/framework.js @@ -17,7 +17,9 @@ var width, height; cameraMat: cameraMat, projMat: camera.projectionMatrix, viewMat: camera.matrixWorldInverse, - models: models + cameraPos: camera.position, + models: models, + screenDim: {w: width, h: height} }); }; @@ -66,8 +68,9 @@ var width, height; }; var init = function() { - // TODO: For performance measurements, disable debug mode! - var debugMode = true; + // For performance measurements, disable debug mode! + //var debugMode = true; + var debugMode = false; if (debugMode) { $('#debugmodewarning').css('display', 'block'); @@ -133,6 +136,23 @@ var width, height; loadTexture('models/sponza/normal.png').then(function(tex) { m.normap = tex; }); + m.specExp = 10.0; + m.removeChannel = -1; + models.push(m); + }); + }); + + loadModel('models/cube.obj', function(o) { + scene.add(o); + uploadModel(o, function(m) { + loadTexture('models/sponza/color.jpg').then(function(tex) { + m.colmap = tex; + }); + loadTexture('models/sponza/normal.png').then(function(tex) { + m.normap = tex; + }); + m.specExp = 100000.0; + m.removeChannel = 2; models.push(m); }); }); diff --git a/js/ui.js b/js/ui.js index 05c1852..07a753b 100644 --- a/js/ui.js +++ b/js/ui.js @@ -7,7 +7,9 @@ var cfg; // TODO: Define config fields and defaults here this.debugView = -1; this.debugScissor = false; - this.enableEffect0 = false; + this.scissoring = true; + this.improvedAABB = true; + this.effects = -1; }; var init = function() { @@ -15,7 +17,7 @@ var cfg; var gui = new dat.GUI(); // TODO: Define any other possible config values - gui.add(cfg, 'debugView', { + /*gui.add(cfg, 'debugView', { 'None': -1, '0 Depth': 0, '1 Position': 1, @@ -23,11 +25,24 @@ var cfg; '3 Color map': 3, '4 Normal map': 4, '5 Surface normal': 5 + });*/ + + gui.add(cfg, 'debugView', { + 'None': -1, + '0 Depth': 0, + '1 Position': 1, + '2 Surface normal': 2, + '3 Color map': 3 }); gui.add(cfg, 'debugScissor'); - var eff0 = gui.addFolder('EFFECT NAME HERE'); - eff0.add(cfg, 'enableEffect0'); + gui.add(cfg, 'scissoring'); + gui.add(cfg, 'improvedAABB'); + gui.add(cfg, 'effects', { + 'None': -1, + '0 Bloom': 0, + '1 Toon': 1 + }); // TODO: add more effects toggles and parameters here }; diff --git a/js/util.js b/js/util.js index 3119dd8..1575347 100644 --- a/js/util.js +++ b/js/util.js @@ -126,6 +126,9 @@ window.readyModelForDraw = function(prog, m) { gl.vertexAttribPointer(prog.a_uv, 2, gl.FLOAT, false, 0, 0); } + gl.uniform1f(prog.u_specExp, m.specExp); + gl.uniform1f(prog.u_remove, m.removeChannel); + gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, m.idx); }; @@ -181,6 +184,50 @@ window.getScissorForLight = (function() { }; })(); +window.getScissorForLightI = (function() { + // Pre-allocate for performance - avoids additional allocation + var a = new THREE.Vector4(0, 0, 0, 0); + var b = new THREE.Vector4(0, 0, 0, 0); + var minpt = new THREE.Vector2(0, 0); + var maxpt = new THREE.Vector2(0, 0); + var ret = [0, 0, 0, 0]; + + return function(view, proj, l) { + a.fromArray(l.pos); + a.w = 1; + a.applyMatrix4(view); + a.x -= l.rad; + a.y -= l.rad; + + b.set(a.x+l.rad*2, a.y+l.rad*2, a.z, a.w); + + // front bottom-left corner of sphere's bounding cube + a.applyMatrix4(proj); + a.divideScalar(a.w); + + // front bottom-right corner of sphere's bounding cube + b.applyMatrix4(proj); + b.divideScalar(b.w); + + minpt.set(Math.max(-1, a.x), Math.max(-1, a.y)); + maxpt.set(Math.min( 1, b.x), Math.min( 1, b.y)); + + if (maxpt.x < -1 || 1 < minpt.x || + maxpt.y < -1 || 1 < minpt.y) { + return null; + } + + minpt.addScalar(1.0); minpt.multiplyScalar(0.5); + maxpt.addScalar(1.0); maxpt.multiplyScalar(0.5); + + ret[0] = Math.round(width * minpt.x); + ret[1] = Math.round(height * minpt.y); + ret[2] = Math.round(width * (maxpt.x - minpt.x)); + ret[3] = Math.round(height * (maxpt.y - minpt.y)); + return ret; + }; +})(); + window.abortIfFramebufferIncomplete = function(fbo) { gl.bindFramebuffer(gl.FRAMEBUFFER, fbo); var fbstatus = gl.checkFramebufferStatus(gl.FRAMEBUFFER);