diff --git a/README.md b/README.md index f810e88..172b35e 100644 --- a/README.md +++ b/README.md @@ -3,424 +3,99 @@ WebGL Deferred Shading **University of Pennsylvania, CIS 565: GPU Programming and Architecture, Project 6** -* (TODO) YOUR NAME HERE -* Tested on: (TODO) **Google Chrome 222.2** on - Windows 22, i7-2222 @ 2.22GHz 22GB, GTX 222 222MB (Moore 2222 Lab) +* Kangning Li +* Tested on: **Google Chrome 46.0.2490.80** on + Ubuntu 14.04, i5-3320M @ 2.60GHz 8GB, Intel HD 4000 (Personal laptop) ### Live Online -[![](img/thumb.png)](http://TODO.github.io/Project6-WebGL-Deferred-Shading) +[![](img/thumb.png)](http://likangning93.github.io/Project6-WebGL-Deferred-Shading) ### Demo Video -[![](img/video.png)](TODO) +[![](img/video.png)](https://vimeo.com/144582823) -### (TODO: Your README) +This repository contains a WebGL deferred shader with the following features: +- deferred shading using WEBGL_draw_buffers +- a toggleable "compressed g-buffer" pipeline +- toggleable scissor testing for both the compressed and uncompressed pipelines +- a tile-based lighting pipeline +- toon shading +- bloom as a post-processing effect -*DO NOT* leave the README to the last minute! It is a crucial part of the -project, and we will not be able to grade you without a good README. +Running the demo above requires support for `OES_texture_float`, `OES_texture_float_linear`, `WEBGL_depth_texture`, and `WEBGL_draw_buffers`. You can check your support on [WebGL Report](http://webglreport.com/). -This assignment has a considerable amount of performance analysis compared -to implementation work. Complete the implementation early to leave time! +### Deferred Shading Overview +The standard deferred shader in this project renders data (position, normals, sampled color, depth, etc.) about what is visible in the scene to a set of WebGL textures referred to as g-buffers. These can be viewed in the debugView settings in the demo. These textures are then passed to a lighting shader that only performs lighting calculations on what is visible in the scene. Finally, a post processing shader can add effects like bloom (implemented here) or efficient depth-of-field simulation and toon shading edge detection if g-buffers are also passed in. +### G-buffer compression +The default pipeline uses 4 g-buffers of vec4s to pass scene information to the lighting shader, along with a buffer for depth: +- position +- normal provided by the scene geometry +- texture mapped color +- texture mapped normal -Instructions (delete me) -======================== +The "compressed" pipeline instead uses 2 g-buffers along with depth: +- texture mapped color +- "compressed" 2-component normal vector (computed from texture mapped and geometry) -This is due at midnight on the evening of Tuesday, October 27. +This compression and decompression of the normal depends on the normal being unit length, which lets the lighting shader compute the magnitude of the normal`s `z` component from its `x` and `y` components. The cardinality of the `z` component is sent as part of the `y` component by padding. If the `z` component is negative , the `y` component is "padded" with a constant so that its magnitude is greater than 1. The lighting shader then only needs to assess the `y` component`s magnitude to determine the `z` component`s cardinality and correctly rebuild the `y` component. -**Summary:** In this project, you'll be introduced to the basics of deferred -shading and WebGL. You'll use GLSL and WebGL to implement a deferred shading -pipeline and various lighting and visual effects. +The lighting shader also reconstructs the world position of a pixel from its depth and screen coordinates with the current view`s camera matrix. More details on the technique can be found [here](https://mynameismjp.wordpress.com/2009/03/10/reconstructing-position-from-depth/) and [here](http://stackoverflow.com/questions/22360810/reconstructing-world-coordinates-from-depth-buffer-and-arbitrary-view-projection). -**Recommendations:** -Take screenshots as you go. Use them to document your progress in your README! +Using "compressed" g-buffers is essentially a tradeoff between memory access and computation, which is usually ideal for GPU applications as GPUs are better at compute than memory access. Even in this imperfect case, in which the 2-component normals are still stored in a vec4 texture, reducing the number of g-buffers still leads to a noticeable improvement in performance. This performance improvement is apparent even as the number of lights increases, as the both pipelines run the lighting shader once per light. -Read (or at least skim) the full README before you begin, so that you know what -to expect and what to prepare for. +![](img/charts/gbufs.png) -### Running the code +### Scissor test -If you have Python, you should be able to run `server.py` to start a server. -Then, open [`http://localhost:10565/`](http://localhost:10565/) in your browser. +Both the "compressed" and "uncompressed" g-buffer pipelines can optionally restrict the screen-space render area of each light using a scissor test, which in most cases speeds up the lighting shader computation for each light. A screen-space bounding box is computed for each light on the CPU, which then restricts the area that the GPU can draw over. This scissor test also allows us to skip lighting for lights that couldn't possibly be visible in the viewport, which is likely a large part of the performance boost. -This project requires a WebGL-capable web browser with support for -`WEBGL_draw_buffers`. You can check for support on -[WebGL Report](http://webglreport.com/). +However, the scissor test is only really useful for the "general" case, where a light's influence covers a relatively small area of the screen. In the case that a light is very close to the camera, the scissor test becomes less beneficial as the light pass for that particular light will essentially span the entire screen. -Google Chrome seems to work best on all platforms. If you have problems running -the starter code, use Chrome or Chromium, and make sure you have updated your -browser and video drivers. +![](img/charts/scissor.png) -In Moore 100C, both Chrome and Firefox work. -See below for notes on profiling/debugging tools. +### Tile based lighting -Use the screenshot button to save a screenshot. +![](img/tiling.png) -## Requirements +The lighting pipelines mentioned above all compute each light's influence independently and blend the result, essentially placing one draw-call per light, which leads to repeated access to the g-buffers and the framebuffer being drawn to. -**Ask on the mailing list for any clarifications.** +One alternative to this is to inform the shader of the lights that influence an area of the scene and have the shader read the position, normal and color data once and then iterate over the given lights, then write the final accumulated result out to the framebuffer. This technique requires splitting the screen space into tiles and computing a datastructure that the shader can access to check the lights that influence a particular tile. -In this project, you are given code for: +This implementation of a tiled pipeline stores this datastructure in another gbuffer and stores lists of light positions and colors, which limits the number of lights that can influence each tile based on the tile's resolution. Each screen space tile of this datastructure gbuffer is split, with one half holding a list of light colors as a vec4 and the other half holding the lights' positions and radii. The end of the list is demarkated with a colorless light with a negative influence radius and extreme z position, thus showing as a blue pixel in the debug view. Thus, the limit on the number of lights is TILE_SIZE * TILE_SIZE / 2, which for a 32 x 32 tile is still 512 lights. -* Loading OBJ files and color/normal map textures -* Camera control -* Partial implementation of deferred shading including many helper functions +This datastructure is computed on the CPU for each light in order, which becomes a performance bottleneck with more lights. Thus, this implementation includes options in the code to set a lower limit on the number of lights that can influence a tile. However, this can lead to undesireable artifacting, as lights that should prominently influence a tile may be omitted when the tile's light list is being computed. This implementation attempts to slightly alleviate this problem with an option to sort the lights based on their z-depth from the camera before computing the light list, thus reducing artifacting in some brightly lit planes that are closer to the camera and roughly parallel with the image plane. -### Required Tasks +![](img/depth_sort.png) +without tiling, with tiling, and depth sorted -**Before doing performance analysis,** you must disable debug mode by changing -`debugMode` to `false` in `framework.js`. Keep it enabled when developing - it -helps find WebGL errors *much* more easily. +However, this does not completely resolve the problem for scenes that have multiple parallel planes at different depths. -You will need to perform the following tasks: +![](img/bad_depth_sort.png) +without tiling, with tiling, and depth sorted -* Complete the deferred shading pipeline so that the Blinn-Phong and Post1 - shaders recieve the correct input. Go through the Starter Code Tour **before - continuing!** +![](img/charts/tiling_vs.png) +(all data measured using 32x32 tiles) -**Effects:** +Tiling leads to noticeable performance improvements in scenes with large numbers of lights. However, again, the computation of the tile datastructure on the CPU is a major performance bottleneck with scenes that have fewer lights but many tiles. -* Implement deferred Blinn-Phong shading (diffuse + specular) - * With normal mapping (code provided) +![](img/charts/tile_size.png) -* Implement one of the following effects: - * Bloom using post-process blur (box or Gaussian) [1] - * Toon shading (with ramp shading + simple depth-edge detection for outlines) +### Bloom -**Optimizations:** +![](img/bloom.png) -* Scissor test optimization: when accumulating shading from each point - light source, only render in a rectangle around the light. - * Show a debug view for this (showing scissor masks clearly), e.g. by - modifying and using `red.frag.glsl` with additive blending and alpha = 0.1. - * Code is provided to compute this rectangle for you, and there are - comments at the relevant place in `deferredRender.js` with more guidance. +Bloom works by sampling the area around each pixel of the lighting pass's output for pixels with a luminance greater than 1, indicating an area that is "brightly lit." Because it is implemented as a post-processing step, it works across all lighting pipelines and adds a constant performance hit over the number of lights in the scene. -* Optimized g-buffer format - reduce the number and size of g-buffers: - * Ideas: - * Pack values together into vec4s - * Use 2-component normals - * Quantize values by using smaller texture types instead of gl.FLOAT - * Reduce number of properties passed via g-buffer, e.g. by: - * Applying the normal map in the `copy` shader pass instead of - copying both geometry normals and normal maps - * Reconstructing world space position using camera matrices and X/Y/depth - * For credit, you must show a good optimization effort and record the - performance of each version you test, in a simple table. - * It is expected that you won't need all 4 provided g-buffers for a basic - pipeline - make sure you disable the unused ones. - * See mainly: `copy.frag.glsl`, `deferred/*.glsl`, `deferredSetup.js` +![](img/charts/bloom.png) -### Extra Tasks +### Toon Shading -You must do at least **10 points** worth of extra features (effects or -optimizations/analysis). +![](img/toon.png) -**Effects:** +Toon shading is accomplished in a lighting pass as a variant on the standard, 4-gbuffer, nontiled pipeline. Similar to bloom, toon shading works by sampling at each screen coordinate the surrounding coordinates. However, it does this with the normal g-buffer and depth buffer, allowing depth-based and normal based edge detection. It also computes a ramp for the blinn-phong lighting computation. -* (3pts) The effect you didn't choose above (bloom or toon shading) +The cost of this additional sampling is multiplied by the fact that the toon shader must be called per light, making toon shading and especially edge detection in the lighting pass very expensive in terms of render time. The above scene, which contains 20 lights, could render in about 80 ms normally (no scissor test) but would take about 185 ms with toon shading. Rendering with the scissor test provides an enormous improvement by eliminating unnecessary lights. However, the performance difference is still about 40 ms versus 55 ms. This difference from the scissor test, however, further illustrates that this implementation of toon shading gets even more expensive as more lights are added. -* (3pts) Screen-space motion blur (blur along velocity direction) [3] - -* (2pts) Allow variability in additional material properties - * Include other properties (e.g. specular coeff/exponent) in g-buffers - * Use this to render objects with different material properties - * These may be uniform across one model draw call, but you'll have to show - multiple models - -**Optimizations/Analysis:** - -* (2pts) Improved screen-space AABB for scissor test - (smaller/more accurate than provided - but beware of CPU/GPU tradeoffs) - -* (3pts) Two-pass **Gaussian** blur using separable convolution (using a second - postprocess render pass) to improve bloom or other 2D blur performance - -* (4-6pts) Light proxies - * (4pts) Instead of rendering a scissored full-screen quad for every light, - render some proxy geometry which covers the part of the screen affected by - the light (e.g. a sphere, for an attenuated point light). - * A model called `sphereModel` is provided which can be drawn in the same - way as the code in `drawScene`. (Must be drawn with a vertex shader which - scales it to the light radius and translates it to the light position.) - * (+2pts) To avoid lighting geometry far behind the light, render the proxy - geometry (e.g. sphere) using an inverted depth test - (`gl.depthFunc(gl.GREATER)`) with depth writing disabled (`gl.depthMask`). - This test will pass only for parts of the screen for which the backside of - the sphere appears behind parts of the scene. - * Note that the copy pass's depth buffer must be bound to the FBO during - this operation! - * Show a debug view for this (showing light proxies) - * Compare performance of this, naive, and scissoring. - -* (8pts) Tile-based deferred shading with detailed performance comparison - * On the CPU, check which lights overlap which tiles. Then, render each tile - just once for all lights (instead of once for each light), applying only - the overlapping lights. - * The method is described very well in - [Yuqin & Sijie's README](https://github.com/YuqinShao/Tile_Based_WebGL_DeferredShader/blob/master/README.md#algorithm-details). - * This feature requires allocating the global light list and tile light - index lists as shown at this link. These can be implemented as textures. - * Show a debug view for this (number of lights per tile) - -* (6pts) Deferred shading without multiple render targets - (i.e. without WEBGL_draw_buffers). - * Render the scene once for each target g-buffer, each time into a different - framebuffer object. - * Include a detailed performance analysis, comparing with/without - WEBGL_draw_buffers (like in the - [Mozilla blog article](https://hacks.mozilla.org/2014/01/webgl-deferred-shading/)). - -* (2-6pts) Compare performance to equivalently-lit forward-rendering: - * (2pts) With no forward-rendering optimizations - * (+2pts) Coarse, per-object back-to-front sorting of geometry for early-z - * (Of course) must render many objects to test - * (+2pts) Z-prepass for early-z - -This extra feature list is not comprehensive. If you have a particular idea -that you would like to implement, please **contact us first** (preferably on -the mailing list). - -**Where possible, all features should be switchable using the GUI panel in -`ui.js`.** - -### Performance & Analysis - -**Before doing performance analysis,** you must disable debug mode by changing -`debugMode` to `false` in `framework.js`. Keep it enabled when developing - it -helps find WebGL errors *much* more easily. - -Optimize your JavaScript and/or GLSL code. Web Tracing Framework -and Chrome/Firefox's profiling tools (see Resources section) will -be useful for this. For each change -that improves performance, show the before and after render times. - -For each new *effect* feature (required or extra), please -provide the following analysis: - -* Concise overview write-up of the feature. -* Performance change due to adding the feature. - * If applicable, how do parameters (such as number of lights, etc.) - affect performance? Show data with simple graphs. -* If you did something to accelerate the feature, what did you do and why? -* How might this feature be optimized beyond your current implementation? - -For each *performance* feature (required or extra), please provide: - -* Concise overview write-up of the feature. -* Detailed performance improvement analysis of adding the feature - * What is the best case scenario for your performance improvement? What is - the worst? Explain briefly. - * Are there tradeoffs to this performance feature? Explain briefly. - * How do parameters (such as number of lights, tile size, etc.) affect - performance? Show data with graphs. - * Show debug views when possible. - * If the debug view correlates with performance, explain how. - -Note: Be aware that stats.js may give 0 millisecond frame timings in Chrome on -occasion - if this happens, you can use the FPS counter. - -### Starter Code Tour - -You'll be working mainly in `deferredRender.js` using raw WebGL. Three.js is -included in the project for various reasons. You won't use it for much, but its -matrix/vector types may come in handy. - -It's highly recommended that you use the browser debugger to inspect variables -to get familiar with the code. At any point, you can also -`console.log(some_var);` to show it in the console and inspect it. - -The setup in `deferredSetup` is already done for you, for many of the features. -If you want to add uniforms (textures or values), you'll change them here. -Therefore, it is recommended that you review the comments to understand the -process, BEFORE starting work in `deferredRender`. - -In `deferredRender`, start at the **START HERE!** comment. -Work through the appropriate `TODO`s as you go - most of them are very -small. Test incrementally (after implementing each part, instead of testing -all at once). -* (The first thing you should be doing is implementing the fullscreen quad!) - -Your _next_ first goal should be to get the debug views working. -Add code in `debug.frag.glsl` to examine your g-buffers before trying to -render them. (Set the debugView in the UI to show them.) - -For editing JavaScript, you can use a simple editor with syntax highlighting -such as Sublime, Vim, Emacs, etc., or the editor built into Chrome. - -* `js/`: JavaScript files for this project. - * `main.js`: Handles initialization of other parts of the program. - * `framework.js`: Loads the scene, camera, etc., and calls your setup/render - functions. Hopefully, you won't need to change anything here. - * `deferredSetup.js`: Deferred shading pipeline setup code. - * `createAndBind(Depth/Color)TargetTexture`: Creates empty textures for - binding to frame buffer objects as render targets. - * `deferredRender.js`: Your deferred shading pipeline execution code. - * `renderFullScreenQuad`: Renders a full-screen quad with the given shader - program. - * `ui.js`: Defines the UI using - [dat.GUI](https://workshop.chromeexperiments.com/examples/gui/). - * The global variable `cfg` can be accessed anywhere in the code to read - configuration values. - * `utils.js`: Utilities for JavaScript and WebGL. - * `abort`: Aborts the program and shows an error. - * `loadTexture`: Loads a texture from a URL into WebGL. - * `loadShaderProgram`: Loads shaders from URLs into a WebGL shader program. - * `loadModel`: Loads a model into WebGL buffers. - * `readyModelForDraw`: Configures the WebGL state to draw a model. - * `drawReadyModel`: Draws a model which has been readied. - * `getScissorForLight`: Computes an approximate scissor rectangle for a - light in world space. -* `glsl/`: GLSL code for each part of the pipeline: - * `clear.*.glsl`: Clears each of the `NUM_GBUFFERS` g-buffers. - * `copy.*.glsl`: Performs standard rendering without any fragment shading, - storing all of the resulting values into the `NUM_GBUFFERS` g-buffers. - * `quad.vert.glsl`: Minimal vertex shader for rendering a single quad. - * `deferred.frag.glsl`: Deferred shading pass (for lighting calculations). - Reads from each of the `NUM_GBUFFERS` g-buffers. - * `post1.frag.glsl`: First post-processing pass. -* `lib/`: JavaScript libraries. -* `models/`: OBJ models for testing. Sponza is the default. -* `index.html`: Main HTML page. -* `server.bat` (Windows) or `server.py` (OS X/Linux): - Runs a web server at `localhost:10565`. - -### The Deferred Shading Pipeline - -See the comments in `deferredSetup.js`/`deferredRender.js` for low-level guidance. - -In order to enable and disable effects using the GUI, upload a vec4 uniform -where each component is an enable/disable flag. In JavaScript, the state of the -UI is accessible anywhere as `cfg.enableEffect0`, etc. - -**Pass 1:** Renders the scene geometry and its properties to the g-buffers. -* `copy.vert.glsl`, `copy.frag.glsl` -* The framebuffer object `pass_copy.fbo` must be bound during this pass. -* Renders into `pass_copy.depthTex` and `pass_copy.gbufs[i]`, which need to be - attached to the framebuffer. - -**Pass 2:** Performs lighting and shading into the color buffer. -* `quad.vert.glsl`, `deferred/blinnphong-pointlight.frag.glsl` -* Takes the g-buffers `pass_copy.gbufs`/`depthTex` as texture inputs to the - fragment shader, on uniforms `u_gbufs` and `u_depth`. -* `pass_deferred.fbo` must be bound. -* Renders into `pass_deferred.colorTex`. - -**Pass 3:** Performs post-processing. -* `quad.vert.glsl`, `post/one.frag.glsl` -* Takes `pass_BlinnPhong_PointLight.colorTex` as a texture input `u_color`. -* Renders directly to the screen if there are no additional passes. - -More passes may be added for additional effects (e.g. combining bloom with -motion blur) or optimizations (e.g. two-pass Gaussian blur for bloom) - -#### Debugging - -If there is a WebGL error, it will be displayed on the developer console and -the renderer will be aborted. To find out where the error came from, look at -the backtrace of the error (you may need to click the triangle to expand the -message). The line right below `wrapper @ webgl-debug.js` will point to the -WebGL call that failed. - -#### Changing the number of g-buffers - -Note that the g-buffers are just `vec4`s - you can put any values you want into -them. However, if you want to change the total number of g-buffers (add more -for additional effects or remove some for performance), you will need to make -changes in a number of places: - -* `deferredSetup.js`/`deferredRender.js`: search for `NUM_GBUFFERS` -* `copy.frag.glsl` -* `deferred.frag.glsl` -* `clear.frag.glsl` - - -## Resources - -* [1] Bloom: - [GPU Gems, Ch. 21](http://http.developer.nvidia.com/GPUGems/gpugems_ch21.html) -* [2] Screen-Space Ambient Occlusion: - [Floored Article](http://floored.com/blog/2013/ssao-screen-space-ambient-occlusion.html) -* [3] Post-Process Motion Blur: - [GPU Gems 3, Ch. 27](http://http.developer.nvidia.com/GPUGems3/gpugems3_ch27.html) - -**Also see:** The articles linked in the course schedule. - -### Profiling and debugging tools - -Built into Firefox: -* Canvas inspector -* Shader Editor -* JavaScript debugger and profiler - -Built into Chrome: -* JavaScript debugger and profiler - -Plug-ins: -* (Chrome/Firefox) [Web Tracing Framework](http://google.github.io/tracing-framework/) -* (Chrome) [Shader Editor](https://chrome.google.com/webstore/detail/shader-editor/ggeaidddejpbakgafapihjbgdlbbbpob) - - -Firefox can also be useful - it has a canvas inspector, WebGL profiling and a -shader editor built in. - - -## README - -Replace the contents of this README.md in a clear manner with the following: - -* A brief description of the project and the specific features you implemented. -* At least one screenshot of your project running. -* A 30+ second video of your project running showing all features. - [Open Broadcaster Software](http://obsproject.com) is recommended. - (Even though your demo can be seen online, using multiple render targets - means it won't run on many computers. A video will work everywhere.) -* A performance analysis (described below). - -### Performance Analysis - -See above. - -### GitHub Pages - -Since this assignment is in WebGL, you can make your project easily viewable by -taking advantage of GitHub's project pages feature. - -Once you are done with the assignment, create a new branch: - -`git branch gh-pages` - -Push the branch to GitHub: - -`git push origin gh-pages` - -Now, you can go to `.github.io/` to see your -renderer online from anywhere. Add this link to your README. - -## Submit - -1. Open a GitHub pull request so that we can see that you have finished. - The title should be "Submission: YOUR NAME". - * **ADDITIONALLY:** - In the body of the pull request, include a link to your repository. -2. Send an email to the TA (gmail: kainino1+cis565@) with: - * **Subject**: in the form of `[CIS565] Project N: PENNKEY`. - * Direct link to your pull request on GitHub. - * Estimate the amount of time you spent on the project. - * If there were any outstanding problems, briefly explain. - * **List the extra features you did.** - * Feedback on the project itself, if any. - -### Third-Party Code Policy - -* Use of any third-party code must be approved by asking on our mailing list. -* If it is approved, all students are welcome to use it. Generally, we approve - use of third-party code that is not a core part of the project. For example, - for the path tracer, we would approve using a third-party library for loading - models, but would not approve copying and pasting a CUDA function for doing - refraction. -* Third-party code **MUST** be credited in README.md. -* Using third-party code without its approval, including using another - student's code, is an academic integrity violation, and will, at minimum, - result in you receiving an F for the semester. +This problem could be sidestepped by passing the normal and depth buffers to the post-processing shader for edge detection and contor drawing, thus making the performance hit constant as with bloom. diff --git a/glsl/blinnphong-pointlight-tiled.frag.glsl b/glsl/blinnphong-pointlight-tiled.frag.glsl new file mode 100644 index 0000000..423124a --- /dev/null +++ b/glsl/blinnphong-pointlight-tiled.frag.glsl @@ -0,0 +1,135 @@ +#version 100 +precision highp float; +precision highp int; + +#define NUM_GBUFFERS 5 + +uniform sampler2D u_gbufs[NUM_GBUFFERS]; +uniform sampler2D u_depth; + +uniform vec3 u_camPos; // in world space + +uniform int u_width; +uniform int u_height; +uniform int u_tileSize; + +varying vec2 v_uv; +const float shininess = 16.0; +// no larger than TILE_SIZE - 1 for now, b/c my datastructure is not efficient. +// don't forget to change max lights over in deferredRender.js! +const int MAX_LIGHTS = 128; //31; max lights is capped by tileSize * tileSize / 2. for 32 x 32 tiles, 512 + +vec3 applyNormalMap(vec3 geomnor, vec3 normap) { + normap = normap * 2.0 - 1.0; + vec3 up = normalize(vec3(0.001, 1, 0.001)); + vec3 surftan = normalize(cross(geomnor, up)); + vec3 surfbinor = cross(geomnor, surftan); + return normap.y * surftan + normap.x * surfbinor + normap.z * geomnor; +} + +vec3 blynnPhong(vec3 lightPos, float radius, vec3 lightCol, vec3 pos, vec3 norm, vec3 col) { + vec3 lightDir = normalize(lightPos - pos); + float lightDistance = length(lightPos - pos); + float lambert = max(dot(lightDir, norm), 0.0); + float specular = 0.0; + if (lambert > 0.0) { + vec3 viewDir = normalize(pos - u_camPos); + + // "blinn phong" + vec3 halfDir = normalize(lightDir + viewDir); + float specAngle = max(dot(halfDir, norm), 0.0); + specular = pow(specAngle, shininess); + } + + float attenuation = max(0.0, radius - lightDistance); + + vec3 color = lambert * col * lightCol + specular * lightCol; + color *= attenuation; + return color; +} + +void main() { + vec4 gb0 = texture2D(u_gbufs[0], v_uv); // texture mapped color + vec4 gb1 = texture2D(u_gbufs[1], v_uv); // world space position + vec4 gb2 = texture2D(u_gbufs[2], v_uv); // geometry normal + vec4 gb3 = texture2D(u_gbufs[3], v_uv); // mapped normal + float depth = texture2D(u_depth, v_uv).x; + // TODO: Extract needed properties from the g-buffers into local variables + // These definitions are suggested for starting out, but you will probably want to change them. + vec3 pos = gb1.xyz; // World-space position + vec3 geomnor = gb2.xyz; // Normals of the geometry as defined, without normal mapping + vec3 colmap = gb0.xyz; // The color map - unlit "albedo" (surface color) + vec3 normap = gb3.xyz; // The raw normal map (normals relative to the surface they're on) + vec3 nor = normalize(applyNormalMap(geomnor, normap)); // The true normals as we want to light them - with the normal map applied to the geometry normals (applyNormalMap above) + + // If nothing was rendered to this pixel, set alpha to 0 so that the + // postprocessing step can render the sky color. + if (depth == 1.0) { + gl_FragColor = vec4(0, 0, 0, 0); + return; + } + + // start reading dem lights! + vec4 lightDataStructure = texture2D(u_gbufs[4], v_uv); + + // figure out which tile this is + // compute the number of tiles + int num_tiles_wide = (u_width + u_tileSize - 1) / u_tileSize; + int num_tiles_high = (u_height + u_tileSize - 1) / u_tileSize; + int num_tiles = num_tiles_wide * num_tiles_high; + + // use the uv to compute this's tile coordinates + int tile_x = int(v_uv.x * float(u_width)) / u_tileSize; + int tile_y = int(v_uv.y * float(u_height)) / u_tileSize; + int tile_number = tile_x + tile_y * num_tiles_wide; + + // use to quickly try out light color sampling. if we can do this, we can sample anything. + float tile_uv_x = float(tile_x * u_tileSize) / float(u_width); // corner's pixel coordinates / dimensions + float tile_uv_y = float(tile_y * u_tileSize) / float(u_height); + vec2 tile_uv = vec2(tile_uv_x, tile_uv_y); + + float uv_xStep = 1.0 / float(u_width); + float uv_yStep = 1.0 / float(u_height); + + tile_uv.x += uv_xStep * 0.5; // sample from center of pixel + tile_uv.y += uv_yStep * 0.5; // sample from center of pixel + + vec2 tile_uv_lightCol = vec2(tile_uv); + tile_uv_lightCol.y += uv_yStep * float(u_tileSize / 2); + + vec3 color = vec3(0.0, 0.0, 0.0); + + //float numLights = 0.0; + + int uv_stepCount = 0; + + // compute blinn-phong for each light + for (int i = 0; i < MAX_LIGHTS; i++) { + // sample light data + vec4 lightCol = texture2D(u_gbufs[4], tile_uv_lightCol); + vec4 lightPos = texture2D(u_gbufs[4], tile_uv); + if (lightPos.w < 0.0) { // negative radius + break; // end of list + } + //numLights += 1.0; + + // compute blynn-phong + color += blynnPhong(lightPos.xyz, lightPos.w, lightCol.rgb, pos, nor, colmap); + + // update sampling coordinate + tile_uv.x += uv_xStep; + tile_uv_lightCol.x += uv_xStep; + uv_stepCount++; + + if (uv_stepCount >= u_tileSize) { + tile_uv.x -= float(uv_stepCount) * uv_xStep; + tile_uv_lightCol.x -= float(uv_stepCount) * uv_xStep; + tile_uv.y += uv_yStep; + tile_uv_lightCol.y += uv_yStep; + uv_stepCount = 0; + } + } + gl_FragColor = vec4(color, 1.0); + //gl_FragColor = vec4(vec3(numLights / float(MAX_LIGHTS * 2)), 1.0); + +} \ No newline at end of file diff --git a/glsl/copy.frag.glsl b/glsl/copy.frag.glsl index 0f5f8f7..d413816 100644 --- a/glsl/copy.frag.glsl +++ b/glsl/copy.frag.glsl @@ -12,4 +12,8 @@ varying vec2 v_uv; void main() { // TODO: copy values into gl_FragData[0], [1], etc. + gl_FragData[0] = texture2D(u_colmap, v_uv); + gl_FragData[1] = vec4(v_position,1.0); + gl_FragData[2] = vec4(v_normal, 0.0); // og normal + gl_FragData[3] = texture2D(u_normap, v_uv); // mapped normal } diff --git a/glsl/deferred/ambient.frag.glsl b/glsl/deferred/ambient.frag.glsl index 1fd4647..4b37e68 100644 --- a/glsl/deferred/ambient.frag.glsl +++ b/glsl/deferred/ambient.frag.glsl @@ -16,12 +16,13 @@ void main() { vec4 gb2 = texture2D(u_gbufs[2], v_uv); vec4 gb3 = texture2D(u_gbufs[3], v_uv); float depth = texture2D(u_depth, v_uv).x; - // TODO: Extract needed properties from the g-buffers into local variables + + vec3 colmap = gb0.xyz; // The color map - unlit "albedo" (surface color) if (depth == 1.0) { gl_FragColor = vec4(0, 0, 0, 0); // set alpha to 0 return; } - gl_FragColor = vec4(0.1, 0.1, 0.1, 1); // TODO: replace this + gl_FragColor = vec4(colmap / 5.0, 1); } diff --git a/glsl/deferred/ambientToon.frag.glsl b/glsl/deferred/ambientToon.frag.glsl new file mode 100644 index 0000000..d99aa6a --- /dev/null +++ b/glsl/deferred/ambientToon.frag.glsl @@ -0,0 +1,60 @@ + +#version 100 +precision highp float; +precision highp int; + +#define NUM_GBUFFERS 4 + +uniform sampler2D u_gbufs[NUM_GBUFFERS]; +uniform sampler2D u_depth; + +varying vec2 v_uv; + +const int lineCheckRadius = 5; +const float lineCheckStep = 0.001; +const float lineCheckDepthRange = 0.01; +const float lineCheckAngleRange = 0.5; + +void main() { + vec4 gb0 = texture2D(u_gbufs[0], v_uv); + vec4 gb1 = texture2D(u_gbufs[1], v_uv); + vec4 gb2 = texture2D(u_gbufs[2], v_uv); + vec4 gb3 = texture2D(u_gbufs[3], v_uv); + float depth = texture2D(u_depth, v_uv).x; + vec3 norm = gb2.xyz; // The true normals as we want to light them - with the normal map applied to the geometry normals (applyNormalMap above) + + vec3 colmap = gb0.xyz; // The color map - unlit "albedo" (surface color) + + if (depth == 1.0) { + gl_FragColor = vec4(0, 0, 0, 0); // set alpha to 0 + return; + } + + vec3 color = vec3(colmap / 5.0); + + // use convolution to add outline based on depth change edge detect + vec2 sampleUV = v_uv - vec2(lineCheckStep * (5.0 / 2.0)); + float numPixelSamples = 1.0; + for (int x = 0; x < lineCheckRadius; x++) { + for (int y = 0; y < lineCheckRadius; y++) { + + float sampleDepth = texture2D(u_depth, sampleUV).x; + + // if sampleDepth is sufficiently different from this fragment's depth, + // darken this pixel as an edge + if (abs(sampleDepth - depth) > lineCheckDepthRange) { + color = vec3(0.0, 0.0, 0.0); + } + + vec3 sampleNorm = texture2D(u_gbufs[2], sampleUV).xyz; + if (dot(sampleNorm, norm) < lineCheckAngleRange) { + color = vec3(0.0, 0.0, 0.0); + } + + } + sampleUV.y = v_uv.y - lineCheckStep * 5.0 / 2.0; + sampleUV.x += lineCheckStep; + } + + gl_FragColor = vec4(color, 1); +} diff --git a/glsl/deferred/blinnphong-pointlight.frag.glsl b/glsl/deferred/blinnphong-pointlight.frag.glsl index b24a54a..0bb6b56 100644 --- a/glsl/deferred/blinnphong-pointlight.frag.glsl +++ b/glsl/deferred/blinnphong-pointlight.frag.glsl @@ -4,11 +4,13 @@ precision highp int; #define NUM_GBUFFERS 4 +uniform vec3 u_camPos; // in world space uniform vec3 u_lightCol; uniform vec3 u_lightPos; uniform float u_lightRad; uniform sampler2D u_gbufs[NUM_GBUFFERS]; uniform sampler2D u_depth; +const float shininess = 16.0; varying vec2 v_uv; @@ -21,12 +23,15 @@ vec3 applyNormalMap(vec3 geomnor, vec3 normap) { } void main() { - vec4 gb0 = texture2D(u_gbufs[0], v_uv); - vec4 gb1 = texture2D(u_gbufs[1], v_uv); - vec4 gb2 = texture2D(u_gbufs[2], v_uv); - vec4 gb3 = texture2D(u_gbufs[3], v_uv); + vec4 gb0 = texture2D(u_gbufs[0], v_uv); // texture mapped color + vec4 gb1 = texture2D(u_gbufs[1], v_uv); // world space position + vec4 gb2 = texture2D(u_gbufs[2], v_uv); // geometry normal + vec4 gb3 = texture2D(u_gbufs[3], v_uv); // mapped normal float depth = texture2D(u_depth, v_uv).x; // TODO: Extract needed properties from the g-buffers into local variables + vec3 pos = gb1.xyz; // cam space position + vec3 colmap = gb0.xyz; // The color map - unlit "albedo" (surface color) + vec3 norm = applyNormalMap(gb2.xyz, gb3.xyz); // The true normals as we want to light them - with the normal map applied to the geometry normals (applyNormalMap above) // If nothing was rendered to this pixel, set alpha to 0 so that the // postprocessing step can render the sky color. @@ -35,5 +40,24 @@ void main() { return; } - gl_FragColor = vec4(0, 0, 1, 1); // TODO: perform lighting calculations + // https://en.wikipedia.org/wiki/Blinn%E2%80%93Phong_shading_model + vec3 lightDir = normalize(u_lightPos - pos); + float lightDistance = length(u_lightPos - pos); + float lambert = max(dot(lightDir, norm), 0.0); + float specular = 0.0; + if (lambert > 0.0) { + vec3 viewDir = normalize(pos - u_camPos); + + // "blinn phong" + vec3 halfDir = normalize(lightDir + viewDir); + float specAngle = max(dot(halfDir, norm), 0.0); + specular = pow(specAngle, shininess); + } + + float attenuation = max(0.0, u_lightRad - lightDistance); + + vec3 color = lambert * colmap * u_lightCol + specular * u_lightCol; + color *= attenuation; + + gl_FragColor = vec4(color, 1); } diff --git a/glsl/deferred/debug.frag.glsl b/glsl/deferred/debug.frag.glsl index 9cbfae4..798ca72 100644 --- a/glsl/deferred/debug.frag.glsl +++ b/glsl/deferred/debug.frag.glsl @@ -21,18 +21,18 @@ vec3 applyNormalMap(vec3 geomnor, vec3 normap) { } void main() { - vec4 gb0 = texture2D(u_gbufs[0], v_uv); - vec4 gb1 = texture2D(u_gbufs[1], v_uv); - vec4 gb2 = texture2D(u_gbufs[2], v_uv); - vec4 gb3 = texture2D(u_gbufs[3], v_uv); + vec4 gb0 = texture2D(u_gbufs[0], v_uv); // texture mapped color + vec4 gb1 = texture2D(u_gbufs[1], v_uv); // world space position + vec4 gb2 = texture2D(u_gbufs[2], v_uv); // geometry normal + vec4 gb3 = texture2D(u_gbufs[3], v_uv); // mapped normal float depth = texture2D(u_depth, v_uv).x; // TODO: Extract needed properties from the g-buffers into local variables // These definitions are suggested for starting out, but you will probably want to change them. - vec3 pos; // World-space position - vec3 geomnor; // Normals of the geometry as defined, without normal mapping - vec3 colmap; // The color map - unlit "albedo" (surface color) - vec3 normap; // The raw normal map (normals relative to the surface they're on) - vec3 nor; // The true normals as we want to light them - with the normal map applied to the geometry normals (applyNormalMap above) + vec3 pos = gb1.xyz; // World-space position + vec3 geomnor = gb2.xyz; // Normals of the geometry as defined, without normal mapping + vec3 colmap = gb0.xyz; // The color map - unlit "albedo" (surface color) + vec3 normap = gb3.xyz; // The raw normal map (normals relative to the surface they're on) + vec3 nor = normalize(applyNormalMap(geomnor, normap)); // The true normals as we want to light them - with the normal map applied to the geometry normals (applyNormalMap above) if (u_debug == 0) { gl_FragColor = vec4(vec3(depth), 1.0); diff --git a/glsl/deferred/toon.frag.glsl b/glsl/deferred/toon.frag.glsl new file mode 100644 index 0000000..5b07348 --- /dev/null +++ b/glsl/deferred/toon.frag.glsl @@ -0,0 +1,105 @@ +#version 100 +precision highp float; +precision highp int; + +#define NUM_GBUFFERS 4 + +uniform vec3 u_camPos; // in world space +uniform vec3 u_lightCol; +uniform vec3 u_lightPos; +uniform float u_lightRad; +uniform sampler2D u_gbufs[NUM_GBUFFERS]; +uniform sampler2D u_depth; +const float shininess = 16.0; + +varying vec2 v_uv; + +const int lineCheckRadius = 5; +const float lineCheckStep = 0.001; +const float lineCheckDepthRange = 0.01; +const float lineCheckAngleRange = 0.5; + +vec3 applyNormalMap(vec3 geomnor, vec3 normap) { + normap = normap * 2.0 - 1.0; + vec3 up = normalize(vec3(0.001, 1, 0.001)); + vec3 surftan = normalize(cross(geomnor, up)); + vec3 surfbinor = cross(geomnor, surftan); + return normap.y * surftan + normap.x * surfbinor + normap.z * geomnor; +} + +float ramp(float value, float steps) { + // clamp value to a step between 0.0 and 1.0. If value > 1.0, do nothing + if (value > 1.0) return value; + float stepSize = 1.0 / steps; + float stepsCovered = floor(value / stepSize); + return stepsCovered * stepSize; +} + +void main() { + vec4 gb0 = texture2D(u_gbufs[0], v_uv); // texture mapped color + vec4 gb1 = texture2D(u_gbufs[1], v_uv); // world space position + vec4 gb2 = texture2D(u_gbufs[2], v_uv); // geometry normal + vec4 gb3 = texture2D(u_gbufs[3], v_uv); // mapped normal + float depth = texture2D(u_depth, v_uv).x; + // TODO: Extract needed properties from the g-buffers into local variables + vec3 pos = gb1.xyz; // cam space position + vec3 colmap = gb0.xyz; // The color map - unlit "albedo" (surface color) + vec3 norm = applyNormalMap(gb2.xyz, gb3.xyz); // The true normals as we want to light them - with the normal map applied to the geometry normals (applyNormalMap above) + + // If nothing was rendered to this pixel, set alpha to 0 so that the + // postprocessing step can render the sky color. + if (depth == 1.0) { + gl_FragColor = vec4(0, 0, 0, 0); + return; + } + + // https://en.wikipedia.org/wiki/Blinn%E2%80%93Phong_shading_model + vec3 lightDir = normalize(u_lightPos - pos); + float lightDistance = length(u_lightPos - pos); + float lambert = max(dot(lightDir, norm), 0.0); + float specular = 0.0; + if (lambert > 0.0) { + vec3 viewDir = normalize(pos - u_camPos); + + // "blinn phong" + vec3 halfDir = normalize(lightDir + viewDir); + float specAngle = max(dot(halfDir, norm), 0.0); + specular = pow(specAngle, shininess); + } + + float attenuation = max(0.0, u_lightRad - lightDistance); + + // clamp lambert and specular components as well as attenuation using a step function + lambert = ramp(lambert, 3.0); + specular = ramp(specular, 3.0); + attenuation = ramp(attenuation, 4.0); + + vec3 color = lambert * colmap * u_lightCol + specular * u_lightCol; + color *= attenuation; + + // use convolution to add outline based on depth change edge detect + vec2 sampleUV = v_uv - vec2(lineCheckStep * (5.0 / 2.0)); + float numPixelSamples = 1.0; + for (int x = 0; x < lineCheckRadius; x++) { + for (int y = 0; y < lineCheckRadius; y++) { + + float sampleDepth = texture2D(u_depth, sampleUV).x; + + // if sampleDepth is sufficiently different from this fragment's depth, + // darken this pixel as an edge + if (abs(sampleDepth - depth) > lineCheckDepthRange) { + color = vec3(0.0, 0.0, 0.0); + } + + vec3 sampleNorm = texture2D(u_gbufs[2], sampleUV).xyz; + if (dot(sampleNorm, norm) < lineCheckAngleRange) { + color = vec3(0.0, 0.0, 0.0); + } + + } + sampleUV.y = v_uv.y - lineCheckStep * 5.0 / 2.0; + sampleUV.x += lineCheckStep; + } + + gl_FragColor = vec4(color, 1); +} diff --git a/glsl/packing/ambientPack.frag.glsl b/glsl/packing/ambientPack.frag.glsl new file mode 100644 index 0000000..49f82ba --- /dev/null +++ b/glsl/packing/ambientPack.frag.glsl @@ -0,0 +1,26 @@ + +#version 100 +precision highp float; +precision highp int; + +#define NUM_GBUFFERS 2 + +uniform sampler2D u_gbufs[NUM_GBUFFERS]; +uniform sampler2D u_depth; + +varying vec2 v_uv; + +void main() { + vec4 gb0 = texture2D(u_gbufs[0], v_uv); + float depth = texture2D(u_depth, v_uv).x; + // TODO: Extract needed properties from the g-buffers into local variables + + vec3 colmap = gb0.xyz; // The color map - unlit "albedo" (surface color) + + if (depth == 1.0) { + gl_FragColor = vec4(0, 0, 0, 0); // set alpha to 0 + return; + } + + gl_FragColor = vec4(colmap / 5.0, 1); // TODO: replace this +} diff --git a/glsl/packing/blinnphong-pointlightPack.frag.glsl b/glsl/packing/blinnphong-pointlightPack.frag.glsl new file mode 100644 index 0000000..7f71561 --- /dev/null +++ b/glsl/packing/blinnphong-pointlightPack.frag.glsl @@ -0,0 +1,79 @@ +#version 100 +precision highp float; +precision highp int; + +#define NUM_GBUFFERS 2 + +uniform vec3 u_camPos; // in world space +uniform vec3 u_lightCol; +uniform vec3 u_lightPos; +uniform float u_lightRad; +uniform sampler2D u_gbufs[NUM_GBUFFERS]; +uniform sampler2D u_depth; +uniform mat4 u_invCameraMat; + +const float shininess = 16.0; + +varying vec2 v_uv; + +void main() { + vec4 gb0 = texture2D(u_gbufs[0], v_uv); // texture mapped color + vec4 gb1 = texture2D(u_gbufs[1], v_uv); // compressed "final" normal + float depth = texture2D(u_depth, v_uv).x; + // TODO: Extract needed properties from the g-buffers into local variables + // These definitions are suggested for starting out, but you will probably want to change them. + vec4 screenPos = vec4(0.0, 0.0, 0.0, 1.0); + vec3 pos = vec3(0.0, 0.0, 0.0); + + if (depth < 1.0) { + // reconstruct screen space position + // https://mynameismjp.wordpress.com/2009/03/10/reconstructing-position-from-depth/ + // http://stackoverflow.com/questions/22360810/reconstructing-world-coordinates-from-depth-buffer-and-arbitrary-view-projection + screenPos.x = v_uv.x * 2.0 - 1.0; + screenPos.y = v_uv.y * 2.0 - 1.0; + screenPos.z = depth * 2.0 - 1.0; + vec4 worldPos = u_invCameraMat * screenPos; + worldPos /= worldPos.w; + pos = worldPos.xyz; + } + + // reconstruct normal + vec3 norm = vec3(gb1.xy, 1.0); + if (abs(norm.x) > 0.0 && abs(norm.y) > 0.0 && abs(norm.y) < 1.0) { + if (abs(norm.x) >= 1.0) { + norm.z = -1.0; + norm.x -= norm.x / abs(norm.x); + } + norm.z *= sqrt(1.0 - norm.x * norm.x - norm.y * norm.y); + } else { + norm = vec3(0.0, 0.0, 0.0); + } + + // If nothing was rendered to this pixel, set alpha to 0 so that the + // postprocessing step can render the sky color. + if (depth == 1.0) { + gl_FragColor = vec4(0, 0, 0, 0); + return; + } + + // https://en.wikipedia.org/wiki/Blinn%E2%80%93Phong_shading_model + vec3 lightDir = normalize(u_lightPos - pos); + float lightDistance = length(u_lightPos - pos); + float lambert = max(dot(lightDir, norm), 0.0); + float specular = 0.0; + if (lambert > 0.0) { + vec3 viewDir = normalize(pos - u_camPos); + + // "blinn phong" + vec3 halfDir = normalize(lightDir + viewDir); + float specAngle = max(dot(halfDir, norm), 0.0); + specular = pow(specAngle, shininess); + } + + float attenuation = max(0.0, u_lightRad - lightDistance); + + vec3 color = lambert * gb0.rgb * u_lightCol + specular * u_lightCol; + color *= attenuation; + + gl_FragColor = vec4(color, 1); +} diff --git a/glsl/packing/clearPack.frag.glsl b/glsl/packing/clearPack.frag.glsl new file mode 100644 index 0000000..9719c57 --- /dev/null +++ b/glsl/packing/clearPack.frag.glsl @@ -0,0 +1,12 @@ +#version 100 +#extension GL_EXT_draw_buffers: enable +precision highp float; +precision highp int; + +#define NUM_GBUFFERS 2 + +void main() { + for (int i = 0; i < NUM_GBUFFERS; i++) { + gl_FragData[i] = vec4(0.0); + } +} diff --git a/glsl/packing/copyPack.frag.glsl b/glsl/packing/copyPack.frag.glsl new file mode 100644 index 0000000..6de2262 --- /dev/null +++ b/glsl/packing/copyPack.frag.glsl @@ -0,0 +1,37 @@ +#version 100 +#extension GL_EXT_draw_buffers: enable +precision highp float; +precision highp int; + +uniform sampler2D u_colmap; +uniform sampler2D u_normap; + +//varying vec3 v_position; +varying vec3 v_normal; +varying vec2 v_uv; + +vec3 applyNormalMap(vec3 geomnor, vec3 normap) { + normap = normap * 2.0 - 1.0; + vec3 up = normalize(vec3(0.001, 1, 0.001)); + vec3 surftan = normalize(cross(geomnor, up)); + vec3 surfbinor = cross(geomnor, surftan); + return normap.y * surftan + normap.x * surfbinor + normap.z * geomnor; +} + +void main() { + // TODO: copy values into gl_FragData[0], [1], etc. + gl_FragData[0] = texture2D(u_colmap, v_uv); + vec4 mappedNormal = texture2D(u_normap, v_uv); + + // compress the normal + if (length(v_normal) > 0.1) { + vec3 finalNormal = normalize(applyNormalMap(v_normal, mappedNormal.xyz)); + // the normal is normalized, which means no element has magnitude greater than 1 + // the fact that it's normalized also means given x and y we can compute z: x ^ 2 + y ^ 2 = 1.0 - z ^ 2 + // so we'll introduce a new invariant: if abs(x) > 1.0, z is negative. else, z is positive. + if (finalNormal.z < 0.0) finalNormal.x += finalNormal.x / abs(finalNormal.x); + gl_FragData[1].xy = finalNormal.xy; + } else { + gl_FragData[1].xy = vec2(0.0, 10.0); // "y" coordinate much bigger than 1 indicates bad norm + } +} diff --git a/glsl/packing/copyPack.vert.glsl b/glsl/packing/copyPack.vert.glsl new file mode 100644 index 0000000..c8a2be5 --- /dev/null +++ b/glsl/packing/copyPack.vert.glsl @@ -0,0 +1,20 @@ +#version 100 +#extension GL_EXT_draw_buffers: enable +precision highp float; +precision highp int; + +uniform mat4 u_cameraMat; + +attribute vec3 a_position; +attribute vec3 a_normal; +attribute vec2 a_uv; + +//varying vec3 v_position; +varying vec3 v_normal; +varying vec2 v_uv; + +void main() { + gl_Position = u_cameraMat * vec4(a_position, 1.0); + v_normal = a_normal; + v_uv = a_uv; +} diff --git a/glsl/packing/debugPack.frag.glsl b/glsl/packing/debugPack.frag.glsl new file mode 100644 index 0000000..547bfc0 --- /dev/null +++ b/glsl/packing/debugPack.frag.glsl @@ -0,0 +1,64 @@ +#version 100 +precision highp float; +precision highp int; + +#define NUM_GBUFFERS 2 + +uniform int u_debug; +uniform sampler2D u_gbufs[NUM_GBUFFERS]; +uniform sampler2D u_depth; +uniform mat4 u_invCameraMat; + +varying vec2 v_uv; + +const vec4 SKY_COLOR = vec4(0.66, 0.73, 1.0, 1.0); + +void main() { + vec4 gb0 = texture2D(u_gbufs[0], v_uv); // texture mapped color + vec4 gb1 = texture2D(u_gbufs[1], v_uv); // screen space position + compressed "final" normal + float depth = texture2D(u_depth, v_uv).x; + // TODO: Extract needed properties from the g-buffers into local variables + // These definitions are suggested for starting out, but you will probably want to change them. + vec4 screenPos = vec4(0.0, 0.0, 0.0, 1.0); + vec3 pos = vec3(0.0, 0.0, 0.0); + + if (depth < 1.0) { + // reconstruct screen space position + // https://mynameismjp.wordpress.com/2009/03/10/reconstructing-position-from-depth/ + // http://stackoverflow.com/questions/22360810/reconstructing-world-coordinates-from-depth-buffer-and-arbitrary-view-projection + screenPos.x = v_uv.x * 2.0 - 1.0; + screenPos.y = v_uv.y * 2.0 - 1.0; + screenPos.z = depth * 2.0 - 1.0; + vec4 worldPos = u_invCameraMat * screenPos; + worldPos /= worldPos.w; + pos = worldPos.xyz; + } + + // reconstruct normal + vec3 norm = vec3(gb1.xy, 1.0); + if (abs(norm.x) > 0.0 && abs(norm.y) > 0.0 && abs(norm.y) < 1.0) { + if (abs(norm.x) >= 1.0) { + norm.z = -1.0; + norm.x -= norm.x / abs(norm.x); + } + norm.z *= sqrt(1.0 - norm.x * norm.x - norm.y * norm.y); + } else { + norm = vec3(0.0, 0.0, 0.0); + } + + if (u_debug == 0) { + gl_FragColor = vec4(vec3(depth), 1.0); + } else if (u_debug == 1) { + gl_FragColor = vec4(abs(pos) * 0.1, 1.0); + } else if (u_debug == 2) { + gl_FragColor = vec4(abs(norm), 1.0); + } else if (u_debug == 3) { + gl_FragColor = vec4(gb0.rgb, 1.0); + } else if (u_debug == 4) { + gl_FragColor = vec4(abs(norm), 1.0); + } else if (u_debug == 5) { + gl_FragColor = vec4(abs(norm), 1.0); + } else { + gl_FragColor = vec4(1, 0, 1, 1); + } +} diff --git a/glsl/post/one.frag.glsl b/glsl/post/one.frag.glsl index 94191cd..7b18c0a 100644 --- a/glsl/post/one.frag.glsl +++ b/glsl/post/one.frag.glsl @@ -2,12 +2,16 @@ precision highp float; precision highp int; +uniform int u_bloom; uniform sampler2D u_color; varying vec2 v_uv; const vec4 SKY_COLOR = vec4(0.01, 0.14, 0.42, 1.0); +const int bloomRadius = 5; +const float bloomStep = 0.001; + void main() { vec4 color = texture2D(u_color, v_uv); @@ -16,5 +20,26 @@ void main() { return; } + if (u_bloom == 1) { + // box blur for bloom + vec2 sampleUV = v_uv - vec2(bloomStep * (5.0 / 2.0)); + vec4 bloomColor = vec4(0.0); + float numPixelSamples = 1.0; + for (int x = 0; x < bloomRadius; x++) { + for (int y = 0; y < bloomRadius; y++) { + vec4 sampleColor = texture2D(u_color, sampleUV); + if (sampleColor.r > 1.0 && sampleColor.g > 1.0 && sampleColor.b > 1.0) { + bloomColor += sampleColor; + } + numPixelSamples += 1.0; + sampleUV.y += bloomStep; + } + sampleUV.y = v_uv.y - bloomStep * 5.0 / 2.0; + sampleUV.x += bloomStep; + } + // add bloom color + color.xyz += bloomColor.xyz / numPixelSamples; + } + gl_FragColor = color; } diff --git a/glsl/red.frag.glsl b/glsl/red.frag.glsl index f8ef1ec..91c1678 100644 --- a/glsl/red.frag.glsl +++ b/glsl/red.frag.glsl @@ -3,5 +3,5 @@ precision highp float; precision highp int; void main() { - gl_FragColor = vec4(1, 0, 0, 1); + gl_FragColor = vec4(1.0, 0, 0, 0.1); } diff --git a/glsl/tiled-numlights-debug.frag.glsl b/glsl/tiled-numlights-debug.frag.glsl new file mode 100644 index 0000000..c27b258 --- /dev/null +++ b/glsl/tiled-numlights-debug.frag.glsl @@ -0,0 +1,80 @@ +#version 100 +precision highp float; +precision highp int; + +#define NUM_GBUFFERS 5 + +uniform sampler2D u_gbufs[NUM_GBUFFERS]; +uniform sampler2D u_depth; + +uniform int u_width; +uniform int u_height; +uniform int u_tileSize; + +varying vec2 v_uv; + +vec3 applyNormalMap(vec3 geomnor, vec3 normap) { + normap = normap * 2.0 - 1.0; + vec3 up = normalize(vec3(0.001, 1, 0.001)); + vec3 surftan = normalize(cross(geomnor, up)); + vec3 surfbinor = cross(geomnor, surftan); + return normap.y * surftan + normap.x * surfbinor + normap.z * geomnor; +} + +void main() { + vec4 gb0 = texture2D(u_gbufs[0], v_uv); // texture mapped color + vec4 gb1 = texture2D(u_gbufs[1], v_uv); // world space position + vec4 gb2 = texture2D(u_gbufs[2], v_uv); // geometry normal + vec4 gb3 = texture2D(u_gbufs[3], v_uv); // mapped normal + float depth = texture2D(u_depth, v_uv).x; + // TODO: Extract needed properties from the g-buffers into local variables + // These definitions are suggested for starting out, but you will probably want to change them. + vec3 pos = gb1.xyz; // World-space position + vec3 geomnor = gb2.xyz; // Normals of the geometry as defined, without normal mapping + vec3 colmap = gb0.xyz; // The color map - unlit "albedo" (surface color) + vec3 normap = gb3.xyz; // The raw normal map (normals relative to the surface they're on) + vec3 nor = normalize(applyNormalMap(geomnor, normap)); // The true normals as we want to light them - with the normal map applied to the geometry normals (applyNormalMap above) + + vec4 lightDataStructure = texture2D(u_gbufs[4], v_uv); + + // figure out which tile this is + // compute the number of tiles + int num_tiles_wide = (u_width + u_tileSize - 1) / u_tileSize; + int num_tiles_high = (u_height + u_tileSize - 1) / u_tileSize; + int num_tiles = num_tiles_wide * num_tiles_high; + + // use the uv to compute this's tile coordinates + int tile_x = int(v_uv.x * float(u_width)) / u_tileSize; + int tile_y = int(v_uv.y * float(u_height)) / u_tileSize; + int tile_number = tile_x + tile_y * num_tiles_wide; + + // use to quickly try out light color sampling. if we can do this, we can sample anything. + float tile_uv_x = float(tile_x * u_tileSize) / float(u_width); // corner's pixel coordinates / dimensions + float tile_uv_y = float(tile_y * u_tileSize) / float(u_height); + vec2 tile_uv = vec2(tile_uv_x, tile_uv_y); + + float uv_xStep = 1.0 / float(u_width); + float uv_yStep = 1.0 / float(u_height); + + tile_uv.x += uv_xStep * 0.5; // sample from center of pixel + tile_uv.y += uv_yStep * 0.5; // sample from center of pixel + + tile_uv.y += uv_yStep * float(u_tileSize / 2); + //tile_uv.x += uv_xStep; + + // for testing sampling + vec4 firstLightCol = texture2D(u_gbufs[4], tile_uv); + gl_FragColor = vec4(firstLightCol.rgb * 0.25, 1.0); + + // for debugging the tile_number calculation + //float gradient = float(tile_number) / float (num_tiles); + //gl_FragColor = vec4(vec3(gradient * 4.0), 1.0); // need multiplier, or gradient per line is too slight + + // for debugging tile_x and tile_y + //gl_FragColor = vec4(vec3(tile_uv_x, tile_uv_y, 0.0), 1.0); + + // for looking at the light datastructure directly + if (abs(lightDataStructure.a) > 0.1) { + gl_FragColor = vec4(lightDataStructure.rgb * 0.5, 1.0); + } +} \ No newline at end of file diff --git a/img/bad_depth_sort.png b/img/bad_depth_sort.png new file mode 100644 index 0000000..cc0a617 Binary files /dev/null and b/img/bad_depth_sort.png differ diff --git a/img/bloom.png b/img/bloom.png new file mode 100644 index 0000000..3131741 Binary files /dev/null and b/img/bloom.png differ diff --git a/img/charts/bloom.csv b/img/charts/bloom.csv new file mode 100644 index 0000000..5c2cb89 --- /dev/null +++ b/img/charts/bloom.csv @@ -0,0 +1,4 @@ +"";"stock";"bloom" +"20";"65";"67" +"40";"116";"117" +"80";"210";"212" diff --git a/img/charts/bloom.png b/img/charts/bloom.png new file mode 100644 index 0000000..1a3c48f Binary files /dev/null and b/img/charts/bloom.png differ diff --git a/img/charts/gbufs.csv b/img/charts/gbufs.csv new file mode 100644 index 0000000..67bcd43 --- /dev/null +++ b/img/charts/gbufs.csv @@ -0,0 +1,5 @@ +"";"4 gbuffers";"2 gbuffers" +"10 lights";"32";"30" +"20 lights";"65";"40" +"40 lights";"110";"82" +"80 lights";"210";"130" diff --git a/img/charts/gbufs.png b/img/charts/gbufs.png new file mode 100644 index 0000000..090e57c Binary files /dev/null and b/img/charts/gbufs.png differ diff --git a/img/charts/scissor.csv b/img/charts/scissor.csv new file mode 100644 index 0000000..1e64020 --- /dev/null +++ b/img/charts/scissor.csv @@ -0,0 +1,5 @@ +"";"4 gbuffers";"2 gbuffers";"4 gbuffers + scissor";"2 gbuffers + scissor" +"10 lights";"32";"30";"15";"16" +"20 lights";"65";"40";"20";"16" +"40 lights";"110";"82";"30";"18" +"80 lights";"210";"130";"49";"28" diff --git a/img/charts/scissor.png b/img/charts/scissor.png new file mode 100644 index 0000000..74a2933 Binary files /dev/null and b/img/charts/scissor.png differ diff --git a/img/charts/tile_size.csv b/img/charts/tile_size.csv new file mode 100644 index 0000000..a01a199 --- /dev/null +++ b/img/charts/tile_size.csv @@ -0,0 +1,6 @@ +"";"tiling" +"16x16";"80" +"32x32";"47" +"64x64";"25" +"128x128";"28" +"256x256";"32" diff --git a/img/charts/tile_size.png b/img/charts/tile_size.png new file mode 100644 index 0000000..a63e8f7 Binary files /dev/null and b/img/charts/tile_size.png differ diff --git a/img/charts/tiling_vs.csv b/img/charts/tiling_vs.csv new file mode 100644 index 0000000..421bd64 --- /dev/null +++ b/img/charts/tiling_vs.csv @@ -0,0 +1,6 @@ +"";"standard";"scissor";"tiling" +"10 lights";"32";"15";"25" +"20 lights";"65";"20";"27" +"40 lights";"110";"30";"30" +"80 lights";"210";"49";"47" +"160 lights";"402";"80";"50" diff --git a/img/charts/tiling_vs.png b/img/charts/tiling_vs.png new file mode 100644 index 0000000..04bbd0a Binary files /dev/null and b/img/charts/tiling_vs.png differ diff --git a/img/depth_sort.png b/img/depth_sort.png new file mode 100644 index 0000000..b0837cc Binary files /dev/null and b/img/depth_sort.png differ diff --git a/img/thumb.png b/img/thumb.png index 9ec8ed0..a8a5025 100644 Binary files a/img/thumb.png and b/img/thumb.png differ diff --git a/img/tiling.png b/img/tiling.png new file mode 100644 index 0000000..e41c1b1 Binary files /dev/null and b/img/tiling.png differ diff --git a/img/toon.png b/img/toon.png new file mode 100644 index 0000000..0c11d14 Binary files /dev/null and b/img/toon.png differ diff --git a/img/video.png b/img/video.png index 9ec8ed0..b63b9d8 100644 Binary files a/img/video.png and b/img/video.png differ diff --git a/js/deferredRender.js b/js/deferredRender.js index b1f238b..0c4061b 100644 --- a/js/deferredRender.js +++ b/js/deferredRender.js @@ -2,6 +2,12 @@ 'use strict'; // deferredSetup.js must be loaded first + var TILE_SIZE = 64; + var MAX_LIGHTS_PER_TILE = TILE_SIZE * TILE_SIZE / 2; + var NUM_TILES_WIDE; + var NUM_TILES_TALL; + var NUM_TILES; + R.deferredRender = function(state) { if (!aborted && ( !R.progCopy || @@ -9,8 +15,17 @@ !R.progClear || !R.prog_Ambient || !R.prog_BlinnPhong_PointLight || + !R.prog_Toon || + !R.prog_ToonAmbient || !R.prog_Debug || - !R.progPost1)) { + !R.progPost1 || + !R.progCopyCompressed || + !R.prog_DebugCompressed || + !R.progClearCompressed || + !R.prog_BlinnPhong_PointLightCompressed || + !R.prog_AmbientCompressed || + !R.prog_DebugTiling || + !R.prog_BlinnPhongTiling)) { console.log('waiting for programs to load...'); return; } @@ -27,85 +42,492 @@ // CHECKITOUT: START HERE! You can even uncomment this: //debugger; - + /* { // TODO: this block should be removed after testing renderFullScreenQuad gl.bindFramebuffer(gl.FRAMEBUFFER, null); // TODO: Implement/test renderFullScreenQuad first renderFullScreenQuad(R.progRed); return; - } + } */ - R.pass_copy.render(state); + // copy pass +if (cfg.enableTiling || cfg.debugTiling) { + R.pass_copy_tile.render(state); + } else if (cfg.compressedGbuffers) { + R.pass_copy_compressed.render(state); + } else { + R.pass_copy.render(state); + } - if (cfg && cfg.debugView >= 0) { + // deferred and post process passes + if (cfg.debugTiling) { + R.pass_debug_tile.render(state); + } + else if (cfg.enableTiling) { + R.pass_deferred_tile.render(state); + R.pass_post1.render(state, R.pass_deferred_tile.colorTex); + } + else if (cfg.compressedGbuffers && cfg.debugView >= 0) { + R.pass_debug_compressed.render(state); + } + else if (cfg.compressedGbuffers) { + R.pass_deferred_compressed.render(state); + R.pass_post1.render(state, R.pass_deferred_compressed.colorTex); + } + else if (cfg && cfg.debugScissor) { + // do a scissor debug render instead of a regular render. + // don't do any post-proccessing in debug mode. + R.pass_debug.debugScissor(state); + } + else if (cfg && cfg.debugView >= 0) { // Do a debug render instead of a regular render // Don't do any post-processing in debug mode R.pass_debug.render(state); - } else { + } + else if (cfg && cfg.enableToon){ + R.pass_toon.render(state); + R.pass_post1.render(state, R.pass_deferred.colorTex); + } + else { // * Deferred pass and postprocessing pass(es) // TODO: uncomment these - //R.pass_deferred.render(state); - //R.pass_post1.render(state); + R.pass_deferred.render(state); + R.pass_post1.render(state, R.pass_deferred.colorTex); // OPTIONAL TODO: call more postprocessing passes, if any } }; + R.bounded = function(point, min, max) { + var boundedX = (min[0] <= point[0]) && (point[0] <= max[0]); + var boundedY = (min[1] <= point[1]) && (point[1] <= max[1]); + return boundedX && boundedY; + } + + // takes boxes like what getScissorForLight returns and returns if + // they overlap or not. + R.boxOverlap = function(box1, box2) { + // the boxes are given as bottom left x, y, width, height + // check each corner of box1 against box2 + var minX = box2[0]; + var minY = box2[1]; + var maxX = box2[0] + box2[2]; + var maxY = box2[1] + box2[3]; + if (R.bounded([box1[0], box1[1]], [minX, minY], [maxX, maxY])) { + return true; + } + if (R.bounded([box1[0] + box1[2], box1[1]], [minX, minY], [maxX, maxY])) { + return true; + } + if (R.bounded([box1[0] + box1[2], box1[1] + box1[3]], [minX, minY], [maxX, maxY])) { + return true; + } + if (R.bounded([box1[0], box1[1] + box1[3]], [minX, minY], [maxX, maxY])) { + return true; + } + // check one corner of box2 against box1 for the "" + minX = box1[0]; + minY = box1[1]; + maxX = box1[0] + box1[2]; + maxY = box1[1] + box1[3]; + if (R.bounded([box2[0], box2[1]], [minX, minY], [maxX, maxY])) { + return true; + } + return false; + } + + /** + * 'copy' pass: Render into g-buffers, update light datastructure + */ + R.pass_copy_tile.render = function(state) { + // * Bind the framebuffer R.pass_copy_tile.fbo + gl.bindFramebuffer(gl.FRAMEBUFFER, R.pass_copy_tile.fbo); + + // * Clear screen using R.progClear + renderFullScreenQuad(R.progClear); + // * Clear depth buffer to value 1.0 using gl.clearDepth and gl.clear + gl.clearDepth(1.0); + // http://webgl.wikia.com/wiki/Clear + gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT); + + // * "Use" the program R.progCopy.prog + gl.useProgram(R.progCopy.prog); + + var m = state.cameraMat.elements; + // * Upload the camera matrix m to the uniform R.progCopy.u_cameraMat + // using gl.uniformMatrix4fv + gl.uniformMatrix4fv(R.progCopy.u_cameraMat, gl.FALSE, m); + + // * Draw the scene + drawScene(state, R.progCopy); + + // update the light datastructure + + if (!NUM_TILES_WIDE || !NUM_TILES_TALL) { + NUM_TILES_WIDE = Math.floor((width + TILE_SIZE - 1) / TILE_SIZE); + NUM_TILES_TALL = Math.floor((height + TILE_SIZE - 1) / TILE_SIZE); + NUM_TILES = NUM_TILES_WIDE * NUM_TILES_TALL; + } + + // preallocate + var lightData = new Float32Array(width * height * 4); + + var lightScissorBox = [0, 0, 0, 0]; + var tileBox = [0, 0, 0, 0]; + + var rowStart = 0; + var rowOffset = 0; + var numLights = 0; + + var lightColIndex = 0; + var lightPosIndex = 0; + + var xi = 0; + var x = 0; + var j = 0; + var lightIdx = 0; + + if (cfg.sortLightsBeforeTiling) { + R.sortLightsByZDepth(state); + } + + var lightColorOffset = width * 4 * (TILE_SIZE / 2); + + // insert the light lists per tile + // for simplicity in indexing, this ONLY works for MAX_LIGHTS < TILE_SIZE + for (var y = 0; y < height; y += TILE_SIZE) { + xi = 0; + for (x = 0; x < width; x += TILE_SIZE) { + // compute start index for this tile + rowStart = xi + (y * width * 4); + rowOffset = 0;; + + xi += TILE_SIZE * 4; + + // compute tile 1's box in pix coordinates. same format as what + // getScissorForLight returns. + // x, y, width, height + // check against each light's scissor box + tileBox = [x, y, TILE_SIZE, TILE_SIZE]; + numLights = 0; + for (j = 0; j < R.NUM_LIGHTS; j++) { + lightIdx = j; + if (cfg.sortLightsBeforeTiling) { + lightIdx = R.lights_z_sorted[j][1]; + } + + // check each light's scissor box against this tile's box + lightScissorBox = getScissorForLight(state.viewMat, + state.projMat, R.lights[lightIdx]); + if (!lightScissorBox) continue; + if (R.boxOverlap(tileBox, lightScissorBox)) { + lightColIndex = rowStart + rowOffset + lightColorOffset; + lightPosIndex = rowStart + rowOffset; + + // insert color + lightData[lightColIndex] = R.lights[lightIdx].col[0]; + lightData[lightColIndex + 1] = R.lights[lightIdx].col[1]; + lightData[lightColIndex + 2] = R.lights[lightIdx].col[2]; + lightData[lightColIndex + 3] = 1; + + // insert radius and direction + lightData[lightPosIndex] = R.lights[lightIdx].pos[0]; + lightData[lightPosIndex + 1] = R.lights[lightIdx].pos[1]; + lightData[lightPosIndex + 2] = R.lights[lightIdx].pos[2]; + lightData[lightPosIndex + 3] = R.lights[lightIdx].rad; + + rowOffset += 4; // lightLists is a bunch of vec4s + numLights++; + + // if necessary, move down a row + if (rowOffset > TILE_SIZE * 4) { + rowOffset = 0; + rowStart += width * 4; + } + } + + if (numLights >= MAX_LIGHTS_PER_TILE) break; + + } + if (numLights < MAX_LIGHTS_PER_TILE) { + // add a NULL light pos/radius (radius = -1) to indicate the end of list + lightData[lightPosIndex] = 0; + lightData[lightPosIndex + 1] = 0; + lightData[lightPosIndex + 2] = 10000000.0; + lightData[lightPosIndex + 3] = -1; + } + + //lightData[lightDataIndex + 2] = 1; // debug + //lightData[lightDataIndex + width * 4 + 2] = 1; // debug + //lightData[lightDataIndex + 3] = 1; // debug + //lightData[lightDataIndex + width * 4 + 3] = 1; // debug + } + } + + gl.bindTexture(gl.TEXTURE_2D, R.pass_copy_tile.gbufs[4]); // light params + gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, width, height, 0, gl.RGBA, + gl.FLOAT, lightData); + gl.bindTexture(gl.TEXTURE_2D, null); + } + + R.sortLightsByZDepth = function(state) { + // build R.lights_z_sorted (unsorted) + var pos_camSpace = new THREE.Vector4(0, 0, 0, 1); + for (var i = 0; i < R.NUM_LIGHTS; i++) { + pos_camSpace.x = R.lights[i].pos[0]; + pos_camSpace.y = R.lights[i].pos[1]; + pos_camSpace.z = R.lights[i].pos[2]; + pos_camSpace.w = 1.0; + pos_camSpace.applyMatrix4(state.viewMat); + pos_camSpace.applyMatrix4(state.projMat); + pos_camSpace.divideScalar(pos_camSpace.w); + // we want negative values (light behind camera) to not even get considered. + if (pos_camSpace.z < 0.0) { + pos_camSpace.z += 10000.0; + } + R.lights_z_sorted[i] = [pos_camSpace.z, i]; + } + + // sort R.lights_z_sorted + R.lights_z_sorted.sort(function(a, b) { + return a[0] - b[0]; + }); + + } + + var bindTexturesForLightPassTiled = function(prog) { + gl.useProgram(prog.prog); + + // * Bind all of the g-buffers and depth buffer as texture uniform + // inputs to the shader + for (var i = 0; i < R.NUM_GBUFFERS + 1; i++) { + if (!prog.u_gbufs[i]) { + gl.activeTexture(gl['TEXTURE' + i]); + gl.bindTexture(gl.TEXTURE_2D, R.pass_copy_tile.depthTex); + gl.uniform1i(prog.u_depth, i); + return; + } + gl.activeTexture(gl['TEXTURE' + i]); + gl.bindTexture(gl.TEXTURE_2D, R.pass_copy_tile.gbufs[i]); + gl.uniform1i(prog.u_gbufs[i], i); + } + gl.activeTexture(gl['TEXTURE' + (R.NUM_GBUFFERS + 1)]); + gl.bindTexture(gl.TEXTURE_2D, R.pass_copy_tile.depthTex); + gl.uniform1i(prog.u_depth, (R.NUM_GBUFFERS + 1)); + }; + + /** + * debug + */ + R.pass_debug_tile.render = function(state) { + // * Unbind any framebuffer, so we can write to the screen + gl.bindFramebuffer(gl.FRAMEBUFFER, null); + + // * Bind/setup the debug "lighting" pass + // * Tell shader which debug view to use + bindTexturesForLightPassTiled(R.prog_DebugTiling); + + gl.uniform1i(R.prog_DebugTiling.u_width, width); + gl.uniform1i(R.prog_DebugTiling.u_height, height); + gl.uniform1i(R.prog_DebugTiling.u_tileSize, TILE_SIZE); + gl.uniform1i(R.prog_DebugTiling.u_numLightsMax, MAX_LIGHTS_PER_TILE); + + // * Render a fullscreen quad to perform shading on + renderFullScreenQuad(R.prog_DebugTiling); + } + + /** + * 'deferred' pass: render all the tiles. no need to accumulate lights! + */ + R.pass_deferred_tile.render = function(state) { // "pass 2" + // * Bind R.pass_deferred_tile.fbo to write into for later postprocessing + gl.bindFramebuffer(gl.FRAMEBUFFER, R.pass_deferred_tile.fbo); + + // * Clear depth to 1.0 and color to black + gl.clearColor(0.0, 0.0, 0.0, 0.0); + gl.clearDepth(1.0); + gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT); + + // Enable blending and use gl.blendFunc to blend with: + // color = 1 * src_color + 1 * dst_color + // goal is to blend each lighting pass into one beautiful frame buffer + // still need to blend in the ambient pass! + gl.enable(gl.BLEND); + gl.blendFunc(gl.ONE, gl.ONE); + + // * Bind/setup the ambient pass, and render using fullscreen quad + bindTexturesForLightPassTiled(R.prog_Ambient); + renderFullScreenQuad(R.prog_Ambient); + + // * Bind/setup the Blinn-Phong pass, and render using fullscreen quad + bindTexturesForLightPassTiled(R.prog_BlinnPhongTiling); + + + gl.uniform3fv(R.prog_BlinnPhongTiling.u_camPos, state.cameraPos.toArray()); + gl.uniform1i(R.prog_BlinnPhongTiling.u_width, width); + gl.uniform1i(R.prog_BlinnPhongTiling.u_height, height); + gl.uniform1i(R.prog_BlinnPhongTiling.u_tileSize, TILE_SIZE); + gl.uniform1i(R.prog_BlinnPhongTiling.u_numLightsMax, MAX_LIGHTS_PER_TILE); + + renderFullScreenQuad(R.prog_BlinnPhongTiling); + + // Disable blending so that it doesn't affect other code + gl.disable(gl.BLEND); + }; + /** * 'copy' pass: Render into g-buffers */ - R.pass_copy.render = function(state) { + R.pass_copy.render = function(state) { // "pass 1" // * Bind the framebuffer R.pass_copy.fbo // TODO: ^ + gl.bindFramebuffer(gl.FRAMEBUFFER, R.pass_copy.fbo); // * Clear screen using R.progClear - TODO: renderFullScreenQuad(R.progClear); + renderFullScreenQuad(R.progClear); // * Clear depth buffer to value 1.0 using gl.clearDepth and gl.clear // TODO: ^ + gl.clearDepth(1.0); // TODO: ^ + // http://webgl.wikia.com/wiki/Clear + gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT); // * "Use" the program R.progCopy.prog // TODO: ^ + gl.useProgram(R.progCopy.prog); + // TODO: Write glsl/copy.frag.glsl var m = state.cameraMat.elements; // * Upload the camera matrix m to the uniform R.progCopy.u_cameraMat // using gl.uniformMatrix4fv // TODO: ^ + gl.uniformMatrix4fv(R.progCopy.u_cameraMat, gl.FALSE, m); // * Draw the scene - drawScene(state); + drawScene(state, R.progCopy); + + // testing updating textures with an array during runtime + //gl.bindTexture(gl.TEXTURE_2D, R.pass_copy.gbufs[0]); // texture mapped color + //var hdiv2 = height;//height / 2; + //var data = new Float32Array(width * hdiv2 * 4); + //for (var i = 0; i < width * hdiv2 * 4; i += 4) { + // data[i] = 1.0; + // data[i + 1] = 0.0; + // data[i + 2] = 0.0; + // data[i + 3] = 1.0; + //} + + //gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, width, hdiv2, 0, gl.RGBA, gl.FLOAT, data); + //gl.bindTexture(gl.TEXTURE_2D, null); }; - var drawScene = function(state) { + R.pass_debug.render = function(state) { + // * Unbind any framebuffer, so we can write to the screen + gl.bindFramebuffer(gl.FRAMEBUFFER, null); + + // * Bind/setup the debug "lighting" pass + // * Tell shader which debug view to use + bindTexturesForLightPass(R.prog_Debug); + gl.uniform1i(R.prog_Debug.u_debug, cfg.debugView); + + // * Render a fullscreen quad to perform shading on + renderFullScreenQuad(R.prog_Debug); + }; + + /** + * 'copy' pass: Render into compressed g-buffers + */ + R.pass_copy_compressed.render = function(state) { // "pass 1" + // * Bind the framebuffer R.pass_copy.fbo + gl.bindFramebuffer(gl.FRAMEBUFFER, R.pass_copy_compressed.fbo); + + // * Clear screen using R.progClear + renderFullScreenQuad(R.progClearCompressed); + // * Clear depth buffer to value 1.0 using gl.clearDepth and gl.clear + gl.clearDepth(1.0); + // http://webgl.wikia.com/wiki/Clear + gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT); + + // * "Use" the program R.progCopyCompressed.prog + gl.useProgram(R.progCopyCompressed.prog); + + var m = state.cameraMat.elements; + // * Upload the camera matrix m to the uniform R.progCopyCompressed.u_cameraMat + // using gl.uniformMatrix4fv + gl.uniformMatrix4fv(R.progCopyCompressed.u_cameraMat, gl.FALSE, m); + + // * Draw the scene + drawScene(state, R.progCopyCompressed); + }; + + var drawScene = function(state, prog) { for (var i = 0; i < state.models.length; i++) { var m = state.models[i]; // If you want to render one model many times, note: // readyModelForDraw only needs to be called once. - readyModelForDraw(R.progCopy, m); + readyModelForDraw(prog, m); drawReadyModel(m); } }; - R.pass_debug.render = function(state) { + R.pass_debug_compressed.render = function(state) { // * Unbind any framebuffer, so we can write to the screen gl.bindFramebuffer(gl.FRAMEBUFFER, null); // * Bind/setup the debug "lighting" pass // * Tell shader which debug view to use - bindTexturesForLightPass(R.prog_Debug); - gl.uniform1i(R.prog_Debug.u_debug, cfg.debugView); + bindTexturesForLightPassCompressed(R.prog_DebugCompressed); + gl.uniform1i(R.prog_DebugCompressed.u_debug, cfg.debugView); + + // upload the inverse camera matrix + var invThreejsMat = new THREE.Matrix4(); + invThreejsMat.copy(state.cameraMat); + invThreejsMat.getInverse(invThreejsMat); + + //var ID = new THREE.Matrix4; + //ID.multiplyMatrices(invThreejsMat, state.cameraMat); + + var m = invThreejsMat.elements; + gl.uniformMatrix4fv(R.prog_DebugCompressed.u_invCameraMat, gl.FALSE, m); // * Render a fullscreen quad to perform shading on - renderFullScreenQuad(R.prog_Debug); + renderFullScreenQuad(R.prog_DebugCompressed); }; + R.pass_debug.debugScissor = function(state) { + // * Unbind any framebuffer, so we can write to the screen + gl.bindFramebuffer(gl.FRAMEBUFFER, null); + + renderFullScreenQuad(R.progClear); + // * Clear depth buffer to value 1.0 using gl.clearDepth and gl.clear + gl.clearDepth(1.0); + // http://webgl.wikia.com/wiki/Clear + gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT); + + gl.enable(gl.BLEND); + gl.blendFunc(gl.SRC_ALPHA, gl.ONE_MINUS_SRC_ALPHA); + + gl.enable(gl.SCISSOR_TEST); + var numLights = R.lights.length; + for (var i = 0; i < numLights; i++) { + var sc = getScissorForLight(state.viewMat, state.projMat, R.lights[i]); + if (sc == null) { + continue; + } + gl.scissor(sc[0], sc[1], sc[2], sc[3]); + renderFullScreenQuad(R.progRed); + } + gl.disable(gl.BLEND); + gl.disable(gl.SCISSOR_TEST); + }; + /** * 'deferred' pass: Add lighting results for each individual light */ - R.pass_deferred.render = function(state) { + R.pass_deferred.render = function(state) { // "pass 2" // * Bind R.pass_deferred.fbo to write into for later postprocessing gl.bindFramebuffer(gl.FRAMEBUFFER, R.pass_deferred.fbo); @@ -118,7 +540,10 @@ // Enable blending and use gl.blendFunc to blend with: // color = 1 * src_color + 1 * dst_color + // goal is to blend each lighting pass into one beautiful frame buffer // TODO: ^ + gl.enable(gl.BLEND); + gl.blendFunc(gl.ONE, gl.ONE); // * Bind/setup the ambient pass, and render using fullscreen quad bindTexturesForLightPass(R.prog_Ambient); @@ -130,7 +555,8 @@ // TODO: add a loop here, over the values in R.lights, which sets the // uniforms R.prog_BlinnPhong_PointLight.u_lightPos/Col/Rad etc., // then does renderFullScreenQuad(R.prog_BlinnPhong_PointLight). - + // blinn phong needs to know camera position + gl.uniform3fv(R.prog_BlinnPhong_PointLight.u_camPos, state.cameraPos.toArray()); // TODO: In the lighting loop, use the scissor test optimization // Enable gl.SCISSOR_TEST, render all lights, then disable it. // @@ -139,6 +565,155 @@ // // var sc = getScissorForLight(state.viewMat, state.projMat, light); + if (cfg.enableScissor) { + gl.enable(gl.SCISSOR_TEST); + } + + var numLights = R.lights.length; + for (var i = 0; i < numLights; i++) { + if (cfg.enableScissor) { + var sc = getScissorForLight(state.viewMat, state.projMat, R.lights[i]); + if (sc == null) { + continue; + } + gl.scissor(sc[0], sc[1], sc[2], sc[3]); + } + + gl.uniform3fv(R.prog_BlinnPhong_PointLight.u_lightPos, R.lights[i].pos); + gl.uniform3fv(R.prog_BlinnPhong_PointLight.u_lightCol, R.lights[i].col); + gl.uniform1f(R.prog_BlinnPhong_PointLight.u_lightRad, R.lights[i].rad); + + renderFullScreenQuad(R.prog_BlinnPhong_PointLight); + } + + if (cfg.enableScissor) { + gl.disable(gl.SCISSOR_TEST); + } + + // Disable blending so that it doesn't affect other code + gl.disable(gl.BLEND); + }; + + /** + * 'deferred' pass: Add lighting results for each individual light + */ + R.pass_toon.render = function(state) { // "pass 2" + // * Bind R.pass_deferred.fbo to write into for later postprocessing + gl.bindFramebuffer(gl.FRAMEBUFFER, R.pass_deferred.fbo); + + // * Clear depth to 1.0 and color to black + gl.clearColor(0.0, 0.0, 0.0, 0.0); + gl.clearDepth(1.0); + gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT); + + // * _ADD_ together the result of each lighting pass + + // Enable blending and use gl.blendFunc to blend with: + // color = 1 * src_color + 1 * dst_color + // goal is to blend each lighting pass into one beautiful frame buffer + gl.enable(gl.BLEND); + gl.blendFunc(gl.ONE, gl.ONE); + + // * Bind/setup the ambient pass, and render using fullscreen quad + bindTexturesForLightPass(R.prog_ToonAmbient); + renderFullScreenQuad(R.prog_ToonAmbient); + + // * Bind/setup the Blinn-Phong pass, and render using fullscreen quad + bindTexturesForLightPass(R.prog_Toon); + + gl.uniform3fv(R.prog_Toon.u_camPos, state.cameraPos.toArray()); + + if (cfg.enableScissor) { + gl.enable(gl.SCISSOR_TEST); + } + + var numLights = R.lights.length; + for (var i = 0; i < numLights; i++) { + if (cfg.enableScissor) { + var sc = getScissorForLight(state.viewMat, state.projMat, R.lights[i]); + if (sc == null) { + continue; + } + gl.scissor(sc[0], sc[1], sc[2], sc[3]); + } + + gl.uniform3fv(R.prog_Toon.u_lightPos, R.lights[i].pos); + gl.uniform3fv(R.prog_Toon.u_lightCol, R.lights[i].col); + gl.uniform1f(R.prog_Toon.u_lightRad, R.lights[i].rad); + + renderFullScreenQuad(R.prog_Toon); + } + + if (cfg.enableScissor) { + gl.disable(gl.SCISSOR_TEST); + } + + // Disable blending so that it doesn't affect other code + gl.disable(gl.BLEND); + }; + + /** + * 'deferred' pass: Add lighting results for each individual light + */ + R.pass_deferred_compressed.render = function(state) { // "pass 2" + // * Bind R.pass_deferred.fbo to write into for later postprocessing + gl.bindFramebuffer(gl.FRAMEBUFFER, R.pass_deferred_compressed.fbo); + + // * Clear depth to 1.0 and color to black + gl.clearColor(0.0, 0.0, 0.0, 0.0); + gl.clearDepth(1.0); + gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT); + + // * _ADD_ together the result of each lighting pass + + // Enable blending and use gl.blendFunc to blend with: + // color = 1 * src_color + 1 * dst_color + // goal is to blend each lighting pass into one beautiful frame buffer + gl.enable(gl.BLEND); + gl.blendFunc(gl.ONE, gl.ONE); + + // * Bind/setup the ambient pass, and render using fullscreen quad + bindTexturesForLightPassCompressed(R.prog_AmbientCompressed); + renderFullScreenQuad(R.prog_AmbientCompressed); + + // * Bind/setup the Blinn-Phong pass, and render using fullscreen quad + bindTexturesForLightPassCompressed(R.prog_BlinnPhong_PointLightCompressed); + + gl.uniform3fv(R.prog_BlinnPhong_PointLightCompressed.u_camPos, state.cameraPos.toArray()); + + // upload the inverse camera matrix + var invThreejsMat = new THREE.Matrix4(); + invThreejsMat.copy(state.cameraMat); + invThreejsMat.getInverse(invThreejsMat); + + var m = invThreejsMat.elements; + gl.uniformMatrix4fv(R.prog_BlinnPhong_PointLightCompressed.u_invCameraMat, gl.FALSE, m); + + if (cfg.enableScissor) { + gl.enable(gl.SCISSOR_TEST); + } + + var numLights = R.lights.length; + for (var i = 0; i < numLights; i++) { + if (cfg.enableScissor) { + var sc = getScissorForLight(state.viewMat, state.projMat, R.lights[i]); + if (sc == null) { + continue; + } + gl.scissor(sc[0], sc[1], sc[2], sc[3]); + } + + gl.uniform3fv(R.prog_BlinnPhong_PointLightCompressed.u_lightPos, R.lights[i].pos); + gl.uniform3fv(R.prog_BlinnPhong_PointLightCompressed.u_lightCol, R.lights[i].col); + gl.uniform1f(R.prog_BlinnPhong_PointLightCompressed.u_lightRad, R.lights[i].rad); + + renderFullScreenQuad(R.prog_BlinnPhong_PointLightCompressed); + } + + if (cfg.enableScissor) { + gl.disable(gl.SCISSOR_TEST); + } + // Disable blending so that it doesn't affect other code gl.disable(gl.BLEND); }; @@ -146,6 +721,20 @@ var bindTexturesForLightPass = function(prog) { gl.useProgram(prog.prog); + // testing updating textures with an array during runtime + //gl.bindTexture(gl.TEXTURE_2D, R.pass_copy.gbufs[0]); // texture mapped color + //var hdiv2 = height;//height / 2; + //var data = new Float32Array(width * hdiv2 * 4); + //for (var i = 0; i < width * hdiv2 * 4; i += 4) { + // data[i] = 1.0; + // data[i + 1] = 0.0; + // data[i + 2] = 0.0; + // data[i + 3] = 1.0; + //} + + //gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, width, hdiv2, 0, gl.RGBA, gl.FLOAT, data); + //gl.bindTexture(gl.TEXTURE_2D, null); + // * Bind all of the g-buffers and depth buffer as texture uniform // inputs to the shader for (var i = 0; i < R.NUM_GBUFFERS; i++) { @@ -158,10 +747,25 @@ gl.uniform1i(prog.u_depth, R.NUM_GBUFFERS); }; + var bindTexturesForLightPassCompressed = function(prog) { + gl.useProgram(prog.prog); + + // * Bind all of the g-buffers and depth buffer as texture uniform + // inputs to the shader + for (var i = 0; i < R.NUM_GBUFFERS_COMPRESSED; i++) { + gl.activeTexture(gl['TEXTURE' + i]); + gl.bindTexture(gl.TEXTURE_2D, R.pass_copy_compressed.gbufs[i]); + gl.uniform1i(prog.u_gbufs[i], i); + } + gl.activeTexture(gl['TEXTURE' + R.NUM_GBUFFERS_COMPRESSED]); + gl.bindTexture(gl.TEXTURE_2D, R.pass_copy_compressed.depthTex); + gl.uniform1i(prog.u_depth, R.NUM_GBUFFERS_COMPRESSED); + }; + /** * 'post1' pass: Perform (first) pass of post-processing */ - R.pass_post1.render = function(state) { + R.pass_post1.render = function(state, tex) { // * Unbind any existing framebuffer (if there are no more passes) gl.bindFramebuffer(gl.FRAMEBUFFER, null); @@ -175,10 +779,13 @@ // * Bind the deferred pass's color output as a texture input // Set gl.TEXTURE0 as the gl.activeTexture unit // TODO: ^ + gl.activeTexture(gl.TEXTURE0); // Bind the TEXTURE_2D, R.pass_deferred.colorTex to the active texture unit // TODO: ^ + gl.bindTexture(gl.TEXTURE_2D, tex); // Configure the R.progPost1.u_color uniform to point at texture unit 0 gl.uniform1i(R.progPost1.u_color, 0); + gl.uniform1i(R.progPost1.u_bloom, cfg.enableBloom); // * Render a fullscreen quad to perform shading on renderFullScreenQuad(R.progPost1); @@ -204,13 +811,13 @@ var init = function() { // Create a new buffer with gl.createBuffer, and save it as vbo. - // TODO: ^ + vbo = gl.createBuffer(); // Bind the VBO as the gl.ARRAY_BUFFER - // TODO: ^ + gl.bindBuffer(gl.ARRAY_BUFFER, vbo); // Upload the positions array to the currently-bound array buffer // using gl.bufferData in static draw mode. - // TODO: ^ + gl.bufferData(gl.ARRAY_BUFFER, positions, gl.STATIC_DRAW); }; return function(prog) { @@ -223,16 +830,18 @@ gl.useProgram(prog.prog); // Bind the VBO as the gl.ARRAY_BUFFER - // TODO: ^ + gl.bindBuffer(gl.ARRAY_BUFFER, vbo); + // Enable the bound buffer as the vertex attrib array for // prog.a_position, using gl.enableVertexAttribArray - // TODO: ^ + gl.enableVertexAttribArray(prog.a_position); + // Use gl.vertexAttribPointer to tell WebGL the type/layout for // prog.a_position's access pattern. - // TODO: ^ + gl.vertexAttribPointer(prog.a_position, 3, gl.FLOAT, false, 0, 0); // Use gl.drawArrays (or gl.drawElements) to draw your quad. - // TODO: ^ + gl.drawArrays(gl.TRIANGLE_STRIP, 0, 4); // Unbind the array buffer. gl.bindBuffer(gl.ARRAY_BUFFER, null); diff --git a/js/deferredSetup.js b/js/deferredSetup.js index 115b016..4118378 100644 --- a/js/deferredSetup.js +++ b/js/deferredSetup.js @@ -5,19 +5,44 @@ R.pass_copy = {}; R.pass_debug = {}; R.pass_deferred = {}; + R.pass_toon = {}; R.pass_post1 = {}; + R.pass_post1_compressed = {}; R.lights = []; + R.lights_z_sorted = []; // this is just a list of depth/index tuples R.NUM_GBUFFERS = 4; + // workflow using compressed g buffers + R.pass_copy_compressed = {}; + R.pass_debug_compressed = {}; + R.pass_deferred_compressed = {}; + + // workflow using tiling + R.pass_copy_tile = {}; + R.pass_debug_tile = {}; + R.pass_deferred_tile = {}; + + R.NUM_GBUFFERS_COMPRESSED = 2; + /** * Set up the deferred pipeline framebuffer objects and textures. */ R.deferredSetup = function() { setupLights(); - loadAllShaderPrograms(); - R.pass_copy.setup(); - R.pass_deferred.setup(); + loadAllShaderPrograms(); // load all the shader programs and get attribute/uniform positions + R.pass_copy.setup(); // allocate + R.pass_deferred.setup(); // allocate + + // compressed g-buffers + R.pass_copy_compressed.setup(); // allocate + R.pass_deferred_compressed.setup(); + + // tiling + R.pass_copy_tile.setup(); + R.pass_deferred_tile.setup(); + + console.log("setup complete"); }; // TODO: Edit if you want to change the light initial positions @@ -48,9 +73,35 @@ 1 + Math.random()], rad: R.LIGHT_RADIUS }); + R.lights_z_sorted.push([0.0, 0]); } }; + R.pass_copy_tile.setup = function() { + // * Create the FBO + R.pass_copy_tile.fbo = gl.createFramebuffer(); + // * Create, bind, and store a depth target texture for the FBO + R.pass_copy_tile.depthTex = createAndBindDepthTargetTexture(R.pass_copy_tile.fbo); + + // * Create, bind, and store "color" target textures for the FBO + // these are the g buffers + // also need a gbuf for lights themselves and a gbuf for tile light lists + R.pass_copy_tile.gbufs = []; + var attachments = []; + for (var i = 0; i < R.NUM_GBUFFERS + 2; i++) { + var attachment = gl_draw_buffers['COLOR_ATTACHMENT' + i + '_WEBGL']; + var tex = createAndBindColorTargetTexture(R.pass_copy_tile.fbo, attachment); + R.pass_copy_tile.gbufs.push(tex); + attachments.push(attachment); + } + + // * Check for framebuffer errors + abortIfFramebufferIncomplete(R.pass_copy_tile.fbo); + // * Tell the WEBGL_draw_buffers extension which FBO attachments are + // being used. (This extension allows for multiple render targets.) + gl_draw_buffers.drawBuffersWEBGL(attachments); + } + /** * Create/configure framebuffer between "copy" and "deferred" stages */ @@ -61,6 +112,7 @@ R.pass_copy.depthTex = createAndBindDepthTargetTexture(R.pass_copy.fbo); // * Create, bind, and store "color" target textures for the FBO + // these are the g buffers R.pass_copy.gbufs = []; var attachments = []; for (var i = 0; i < R.NUM_GBUFFERS; i++) { @@ -70,6 +122,8 @@ attachments.push(attachment); } + // this basically attaches a bunch of draw_buffers to the fbo? + // * Check for framebuffer errors abortIfFramebufferIncomplete(R.pass_copy.fbo); // * Tell the WEBGL_draw_buffers extension which FBO attachments are @@ -94,6 +148,69 @@ gl_draw_buffers.drawBuffersWEBGL([gl_draw_buffers.COLOR_ATTACHMENT0_WEBGL]); }; + /** + * Create/configure framebuffer between "deferred" and "post1" stages + */ + R.pass_deferred_compressed.setup = function() { + // * Create the FBO + R.pass_deferred_compressed.fbo = gl.createFramebuffer(); + // * Create, bind, and store a single color target texture for the FBO + R.pass_deferred_compressed.colorTex = createAndBindColorTargetTexture( + R.pass_deferred_compressed.fbo, gl_draw_buffers.COLOR_ATTACHMENT0_WEBGL); + + // * Check for framebuffer errors + abortIfFramebufferIncomplete(R.pass_deferred_compressed.fbo); + // * Tell the WEBGL_draw_buffers extension which FBO attachments are + // being used. (This extension allows for multiple render targets.) + gl_draw_buffers.drawBuffersWEBGL([gl_draw_buffers.COLOR_ATTACHMENT0_WEBGL]); + }; + + R.pass_deferred_tile.setup = function() { + // * Create the FBO + R.pass_deferred_tile.fbo = gl.createFramebuffer(); + // * Create, bind, and store a single color target texture for the FBO + R.pass_deferred_tile.colorTex = createAndBindColorTargetTexture( + R.pass_deferred_tile.fbo, gl_draw_buffers.COLOR_ATTACHMENT0_WEBGL); + + // * Check for framebuffer errors + abortIfFramebufferIncomplete(R.pass_deferred_tile.fbo); + // * Tell the WEBGL_draw_buffers extension which FBO attachments are + // being used. (This extension allows for multiple render targets.) + gl_draw_buffers.drawBuffersWEBGL([gl_draw_buffers.COLOR_ATTACHMENT0_WEBGL]); + }; + + /** + * Create/configure framebuffer between "copy" and "deferred" stages for compressed workflow + */ + R.pass_copy_compressed.setup = function() { + // * Create the FBO + R.pass_copy_compressed.fbo = gl.createFramebuffer(); + // * Create, bind, and store a depth target texture for the FBO + R.pass_copy_compressed.depthTex = createAndBindDepthTargetTexture(R.pass_copy_compressed.fbo); + + // * Create, bind, and store "color" target textures for the FBO + // these are the g buffers + R.pass_copy_compressed.gbufs = []; + var attachments = []; + for (var i = 0; i < R.NUM_GBUFFERS_COMPRESSED; i++) { + // don't want to use the same draw buffers over again? + var attachment = gl_draw_buffers['COLOR_ATTACHMENT' + i + '_WEBGL']; + var tex = createAndBindColorTargetTexture(R.pass_copy_compressed.fbo, attachment); + console.log(attachment); + + R.pass_copy_compressed.gbufs.push(tex); + attachments.push(attachment); + } + + // this basically attaches a bunch of draw_buffers to the fbo? + + // * Check for framebuffer errors + abortIfFramebufferIncomplete(R.pass_copy_compressed.fbo); + // * Tell the WEBGL_draw_buffers extension which FBO attachments are + // being used. (This extension allows for multiple render targets.) + gl_draw_buffers.drawBuffersWEBGL(attachments); + }; + /** * Loads all of the shader programs used in the pipeline. */ @@ -132,14 +249,29 @@ R.prog_Ambient = p; }); + loadDeferredProgram('ambientToon', function(p) { + // Save the object into this variable for access later + R.prog_ToonAmbient = p; + }); + loadDeferredProgram('blinnphong-pointlight', function(p) { // Save the object into this variable for access later + p.u_camPos = gl.getUniformLocation(p.prog, 'u_camPos'); p.u_lightPos = gl.getUniformLocation(p.prog, 'u_lightPos'); p.u_lightCol = gl.getUniformLocation(p.prog, 'u_lightCol'); p.u_lightRad = gl.getUniformLocation(p.prog, 'u_lightRad'); R.prog_BlinnPhong_PointLight = p; }); + loadDeferredProgram('toon', function(p) { + // Save the object into this variable for access later + p.u_camPos = gl.getUniformLocation(p.prog, 'u_camPos'); + p.u_lightPos = gl.getUniformLocation(p.prog, 'u_lightPos'); + p.u_lightCol = gl.getUniformLocation(p.prog, 'u_lightCol'); + p.u_lightRad = gl.getUniformLocation(p.prog, 'u_lightRad'); + R.prog_Toon = p; + }); + loadDeferredProgram('debug', function(p) { p.u_debug = gl.getUniformLocation(p.prog, 'u_debug'); // Save the object into this variable for access later @@ -148,11 +280,130 @@ loadPostProgram('one', function(p) { p.u_color = gl.getUniformLocation(p.prog, 'u_color'); + p.u_bloom = gl.getUniformLocation(p.prog, 'u_bloom'); // Save the object into this variable for access later R.progPost1 = p; }); // TODO: If you add more passes, load and set up their shader programs. + // compressed position/normal workflow + loadShaderProgram(gl, 'glsl/packing/copyPack.vert.glsl', 'glsl/packing/copyPack.frag.glsl', + function(prog) { + // Create an object to hold info about this shader program + var p = { prog: prog }; + + // Retrieve the uniform and attribute locations + p.u_cameraMat = gl.getUniformLocation(prog, 'u_cameraMat'); + p.u_colmap = gl.getUniformLocation(prog, 'u_colmap'); + p.u_normap = gl.getUniformLocation(prog, 'u_normap'); + p.a_position = gl.getAttribLocation(prog, 'a_position'); + p.a_normal = gl.getAttribLocation(prog, 'a_normal'); + p.a_uv = gl.getAttribLocation(prog, 'a_uv'); + + // Save the object into this variable for access later + R.progCopyCompressed = p; + }); + + loadShaderProgram(gl, 'glsl/quad.vert.glsl', 'glsl/packing/debugPack.frag.glsl', + function(prog) { + // Create an object to hold info about this shader program + var p = { prog: prog }; + + // Retrieve the uniform and attribute locations + p.u_gbufs = []; + for (var i = 0; i < R.NUM_GBUFFERS_COMPRESSED; i++) { + p.u_gbufs[i] = gl.getUniformLocation(prog, 'u_gbufs[' + i + ']'); + } + p.u_depth = gl.getUniformLocation(prog, 'u_depth'); + p.a_position = gl.getAttribLocation(prog, 'a_position'); + p.u_debug = gl.getUniformLocation(p.prog, 'u_debug'); + p.u_invCameraMat = gl.getUniformLocation(prog, 'u_invCameraMat'); + + R.prog_DebugCompressed = p; + }); + + loadShaderProgram(gl, 'glsl/quad.vert.glsl', 'glsl/packing/clearPack.frag.glsl', + function(prog) { + // Create an object to hold info about this shader program + R.progClearCompressed = { prog: prog }; + }); + + loadShaderProgram(gl, 'glsl/quad.vert.glsl', + 'glsl/packing/blinnphong-pointlightPack.frag.glsl', + function(prog) { + // Create an object to hold info about this shader program + var p = { prog: prog }; + + // Retrieve the uniform and attribute locations + p.u_gbufs = []; + for (var i = 0; i < R.NUM_GBUFFERS_COMPRESSED; i++) { + p.u_gbufs[i] = gl.getUniformLocation(prog, 'u_gbufs[' + i + ']'); + } + p.u_depth = gl.getUniformLocation(prog, 'u_depth'); + + p.u_camPos = gl.getUniformLocation(p.prog, 'u_camPos'); + p.u_lightPos = gl.getUniformLocation(p.prog, 'u_lightPos'); + p.u_lightCol = gl.getUniformLocation(p.prog, 'u_lightCol'); + p.u_lightRad = gl.getUniformLocation(p.prog, 'u_lightRad'); + p.u_invCameraMat = gl.getUniformLocation(prog, 'u_invCameraMat'); + + R.prog_BlinnPhong_PointLightCompressed = p; + }); + + loadShaderProgram(gl, 'glsl/quad.vert.glsl', + 'glsl/packing/ambientPack.frag.glsl', + function(prog) { + // Create an object to hold info about this shader program + var p = { prog: prog }; + + // Retrieve the uniform and attribute locations + p.u_gbufs = []; + for (var i = 0; i < R.NUM_GBUFFERS_COMPRESSED; i++) { + p.u_gbufs[i] = gl.getUniformLocation(prog, 'u_gbufs[' + i + ']'); + } + p.u_depth = gl.getUniformLocation(prog, 'u_depth'); + + R.prog_AmbientCompressed = p; + }); + + loadShaderProgram(gl, 'glsl/quad.vert.glsl', + 'glsl/tiled-numlights-debug.frag.glsl', + function(prog) { + // Create an object to hold info about this shader program + var p = { prog: prog }; + + // Retrieve the uniform and attribute locations + p.u_gbufs = []; + for (var i = 0; i < R.NUM_GBUFFERS + 1; i++) { + p.u_gbufs[i] = gl.getUniformLocation(prog, 'u_gbufs[' + i + ']'); + } + p.u_depth = gl.getUniformLocation(prog, 'u_depth'); + p.u_width = gl.getUniformLocation(p.prog, 'u_width'); + p.u_height = gl.getUniformLocation(p.prog, 'u_height'); + p.u_tileSize = gl.getUniformLocation(p.prog, 'u_tileSize'); + + R.prog_DebugTiling = p; + }); + + loadShaderProgram(gl, 'glsl/quad.vert.glsl', + 'glsl/blinnphong-pointlight-tiled.frag.glsl', + function(prog) { + // Create an object to hold info about this shader program + var p = { prog: prog }; + + // Retrieve the uniform and attribute locations + p.u_gbufs = []; + for (var i = 0; i < R.NUM_GBUFFERS + 1; i++) { + p.u_gbufs[i] = gl.getUniformLocation(prog, 'u_gbufs[' + i + ']'); + } + p.u_depth = gl.getUniformLocation(prog, 'u_depth'); + p.u_width = gl.getUniformLocation(p.prog, 'u_width'); + p.u_height = gl.getUniformLocation(p.prog, 'u_height'); + p.u_tileSize = gl.getUniformLocation(p.prog, 'u_tileSize'); + p.u_camPos = gl.getUniformLocation(p.prog, 'u_camPos'); + + R.prog_BlinnPhongTiling = p; + }); }; var loadDeferredProgram = function(name, callback) { diff --git a/js/framework.js b/js/framework.js index a0653a5..38d9b36 100644 --- a/js/framework.js +++ b/js/framework.js @@ -17,16 +17,16 @@ var width, height; cameraMat: cameraMat, projMat: camera.projectionMatrix, viewMat: camera.matrixWorldInverse, + cameraPos: camera.position, models: models }); }; var update = function() { controls.update(); + stats.end(); stats.begin(); render(); - gl.finish(); - stats.end(); if (!aborted) { requestAnimationFrame(update); } @@ -67,7 +67,13 @@ var width, height; var init = function() { // TODO: For performance measurements, disable debug mode! - var debugMode = true; + var debugMode = false; + canvas = document.getElementById('canvas'); + renderer = new THREE.WebGLRenderer({ + canvas: canvas, + preserveDrawingBuffer: debugMode + }); + gl = renderer.context; if (debugMode) { $('#debugmodewarning').css('display', 'block'); @@ -78,13 +84,6 @@ var width, height; gl = WebGLDebugUtils.makeDebugContext(gl, throwOnGLError); } - canvas = document.getElementById('canvas'); - renderer = new THREE.WebGLRenderer({ - canvas: canvas, - preserveDrawingBuffer: debugMode - }); - gl = renderer.context; - initExtensions(); stats = new Stats(); diff --git a/js/ui.js b/js/ui.js index 05c1852..f839818 100644 --- a/js/ui.js +++ b/js/ui.js @@ -7,7 +7,13 @@ var cfg; // TODO: Define config fields and defaults here this.debugView = -1; this.debugScissor = false; - this.enableEffect0 = false; + this.enableScissor = false; + this.enableBloom = false; + this.enableToon = false; + this.compressedGbuffers = false; + this.debugTiling = false; + this.enableTiling = false; + this.sortLightsBeforeTiling = false; }; var init = function() { @@ -25,10 +31,15 @@ var cfg; '5 Surface normal': 5 }); gui.add(cfg, 'debugScissor'); + gui.add(cfg, 'enableScissor'); + gui.add(cfg, 'debugTiling'); + gui.add(cfg, 'enableTiling'); + gui.add(cfg, 'sortLightsBeforeTiling'); - var eff0 = gui.addFolder('EFFECT NAME HERE'); - eff0.add(cfg, 'enableEffect0'); - // TODO: add more effects toggles and parameters here + var eff0 = gui.addFolder('effects'); + eff0.add(cfg, 'enableBloom'); + eff0.add(cfg, 'enableToon'); + gui.add(cfg, 'compressedGbuffers'); }; window.handle_load.push(init); diff --git a/js/util.js b/js/util.js index 3119dd8..95baf0e 100644 --- a/js/util.js +++ b/js/util.js @@ -152,7 +152,7 @@ window.getScissorForLight = (function() { a.applyMatrix4(proj); a.divideScalar(a.w); - // front bottom-left corner of sphere's bounding cube + // other corner of sphere's bounding cube b.fromArray(l.pos); b.w = 1; b.applyMatrix4(view); diff --git a/notes.txt b/notes.txt new file mode 100644 index 0000000..b208472 --- /dev/null +++ b/notes.txt @@ -0,0 +1,208 @@ +HOW THINGS WORK +-so deferred shading is all about reducing the number of fragment shadings that need to happen +-so the deferred shader rasterizes, and then performs lighting computations only for what's visible +-in webgl apparently this requires writing out buffers of what's visible +-so these are the "multiple render passes" +-btw, shader programs can have inputs and outputs bound on init + +BLOOM +-something's... not so right +-but the idea of this should work +-just need a better filter system +-should we make this a separate post processing shader? can we do this without addtl buffers? + -yes. probably. hopefully. + -which would make profiling later easier. +-TODO: + -add camera position uniform to blinn-phong [done. turns out this was in Kai's updates] + -find a better filter [done] + -make this a separate post-processing shader + +SCISSOR TEST [done] +-ok ok ok. so it's restricting the shading to a screen space "box" that gets a lighting computation +-shouldn't be too bad +-we'll need to tweak ambient, btw. scene is too dark atm. [done] +-TODO: + -enable and use Kai's primitive scissor test [done] + -add scissor test debug mode [done] + +PACKING AND OPTIMIZATION -> all about trying to do this stuff while keeping it toggleable +-try 2-component normals + -use length of x and y to get magnitude of z + -these are normalized, so no component will be greater than 1 -> use that to pack sign of z + -how to do this while making it toggleable? -> probably needs a new shader + +-apply normal map and pass it along -> easy, probably. but will interfere with debug -> needs a new shader + +-reconstruct world space pos w/cam matrix, x/y/depth -> hmmm. -> needs a new shader + -need to invert cam matrix? -> look up how to do this. + -we probably already need to add a uniform for cam position, b/c blinn phong is broken without it + +2 COMPONENT NORMALS and WORLD SPACE RECONSTRUCT +-so we don't want to change from buffers of vec4s +-which means if we're packing, we want to pack both 2-comp normals and world space pos into one buffer + +-so we'll need a new copy.frag.glsl and a new copy.vert.glsl + -new copy.vert.glsl will need to push screen space position over + -new copy.frag.glsl will only do 2 buffers: texture2D and compacted position/normal + -new copy.frag.glsl will also need to apply surface normal internally + +-we'll need to add a new blinnphong-pointlight.frag.glsl and ambient.frag.glsl as well + -just need to add position/normal reconstruction + +-which means we'll need to add setup for all of these in deferredSetup.js +-might as well add a new debug.frag.glsl as well to help debug reconstruction +-so we're basically duplicating a huge chunk of the project. ok. + +SCREEN SPACE MOTION BLUR and/or TOON SHADING +-screen space motion blur has a GPU gems article, so this might be more straightforward + -would also be post processing +-but toon shading: -> TOON SHADING IS COOL + -need to sample at each frag for depth. if depth change is beyond some threshold, then need to add a line + -ramp shading: like cutting up the gradient of blinn-phong for specular. we can probably generate this. + -so this has to be an alternate shader to blinn-phong + + +TILING +-basically trying to reduce number of light calculations by tiling +-think about how to do this +-seems like it could extend from scissor test? + +TODO before seeing Kai: +-make bloom toggleable [done] +-make scissor test toggleable [done...? there was a mystery crash] +-add a toon shader. make it toggleable. should theoretically be very similar to blinnphong in the js + +Seeing Kai: +-so I'm trying to have a separate pipeline that I can toggle +-this pipeline has copy stuff that produces fewer gbuffers +-my understanding is that each framebuffer can have its own gbuffers +-but the gbuffers here seem to be wrong. what's up? + +TILING FOR REAL +-so the basic idea is to store each light's attributes in textures +-then you store each tile's lights list in another texture +-and to render a tile, you run the shader over that tile region with the index and length of the tile's light list + +QUESTIONS +-how to build the tile/light lists? + -likely involves buffering data? + -ok. lists: can use texImage2D with internal format UNSIGNED_INT, format DEPTH_COMPONENT. presumably we can upload an unsigned int array. + -https://www.khronos.org/registry/webgl/extensions/WEBGL_depth_texture/ + -light info: will probably have to be vec4 for pos/radius and vec4 color + -will need to update per render with new position + -we'll need a way to compute a "new UV" in the shader per light index + -can PROBABLY use texture2D in the shader and pass in an appropriate UV value + +-how to get the lighting shader to loop over lights? + -define some max number of lights? + -glsl for loops have to have a determined number of iters right? +-how to get the lighting shader to only handle the tile it's in? + -scissor? -> yeah yeah yeah! + -NO. we can render all the tiles at once. + -this necessitates a light data per tile list texture + +LOOK AT +-binding textures to uniform pos: util.js -> window.readyModelForDraw + -it's just a matter of binding texture to uniform + -be careful with the whole active texture thing? +-also: deferredRender.js -> bindTexturesForLightPass +-reloading textures: util.js -> handleTextureLoaded + -there's a "texImage2D" use here. + -MSDN says it can be used with int arrays -> ArrayBufferView + -https://developer.mozilla.org/en-US/docs/Web/API/ArrayBufferView + -http://www.javascripture.com/ArrayBufferView + -so apparently ArrayBufferView just means "some array" + -so straight up Uint16 arrays are ok? + -http://stackoverflow.com/questions/9046643/webgl-create-texture + +TRY +-overwriting the color pass texture in bindTexturesForLightPass with a float array +-making a texture the same way as in framework's loadmodels. can we inspect it? + -no + +TODO +-add a texture of light info (color, position + radius) +-add an int texture of light lists -> numLights * numTiles size. biiiiig. +-add an int texture of light list pointers (per tile thingy) -> actually, can we just bind these as uniforms? lol why not? +-add a light shader that does blinn phong over each light in the list + -needs to compute UV from indices provided, so needs to know how many lights + -needs to have enough looping room to do all the lights possible -> uh oh + -introduces a restriction on how many lights we can do per pass + -should we care about blending? or just artificially limit? + -presumably we're hoping you won't have > 100 lights in one tile + -aaaaand all the stupid messy binding and stuff that's associated with this + +new understanding: +-ok. so the light data structure has to be bound to the fbo +-but how is this done?! it seems all textures attached to the fbo must be the same dimensions as the fbo! +-as long as max_lights_per_tile < tile_width, this can still be done... hmmm + +Steps +1) set up a new copy pass for tiling [done, untested] + -include a new gbuf for each element in the datastructure +2) write the new copy pass [done, untested] + -must also overwrite the appropriate gbuffers with light datastructure + -this will heavily change how indexing is handled +3) write a new bindTexturesForLightPass [done] +4) write the debug_tile render pass [done] +5) write a new debug shader [done] + +notes: +-to simplify, we're going to clamp the number of lights to resolution width + -simplifies indexing when reading a light's params +-to reduce lookups: since we get a huge amount of memory for a sparse amount of data, let's have each light list literally store the light data. +-will simplify lookup in shader + +PROBLEMS +-so at the moment, the limit on number of tiles means that some lights very obviously end up missing from tiles +-let's add an option to toggle a different kind of light input: it will sort the lights based on distance along the camera's view and then bin them using that sort + +-wtf is going on with github.io? how do we do stuff there? do we have to host the same code in 2 places on github?! + +TODO: +-add appropriate global structures +-figger out how to z sort -> transformations? durrr hur hur hur +-look at js sort: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/sort + +-so light sorting is only sometimes useful. glad I made it a toggleable + +TODO: [done] -> but there's still some artifacting, possibly remnant of scissor test? +-improve indexing so that we can support more lights per tile. 0.5 * TILESIZE * TILESIZE - 1 is ideal + -this shouldn't be that hard + -we can devote half the tile to colors and half the tile to positions, for better visualization + -will need to remove the fake global light list. who needs it anyway? + +G-buffer format change performance: +-10 lights: 32 ms 30 ms +-20 lights: 65 ms 40 ms +-40 lights: 110 ms 82 ms +-80 lights: 210 ms 130 ms +-160 lights: 402 ms, + +scissor test: +-10 lights: 15 ms 16 ms +-20 lights: 20 ms 16 ms +-40 lights: 30 ms 18 ms +-80 lights: 49 ms 28 ms + +tiling +-10 lights: 25 ms +-20 lights: 27 ms +-40 lights: 30 ms +-80 lights: 47 ms +-160 lights: 402 ms, 80 ms, 50 ms + +80 lights: +16x16: 80 ms +32x32: 47 ms +64x64: 25 ms +128x128: 28 ms +256x256: 32 ms + +bloom, stock pipeline: +20 lights: 65 ms, 67 ms +40 lights: 116 ms, 117 ms +80 lights: 210 ms, 212 ms + + +