Fix bugs, add an end-to-end test with a model with all new features #198
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Adds groups_input_mixin parameter to control grouped convolutions in the input_mixin Conv1x1 layer. The parameter is propagated through Layer, LayerArrayParams, and LayerArray constructors. Factory parsing defaults to 1 if not specified in the model JSON for backward compatibility. Also fixes a bug in test_real_time_safe where make_layer_all_films was incorrectly activating head1x1_post_film when head1x1 was inactive.
Aligns the JSON configuration key with the LayerArrayParams attribute name for consistency. The factory now reads 'groups_input' instead of 'groups' from the layer configuration.
…ilm_params Removed the separate gating_activation_post_film_params parameter and now use activation_post_film_params for both gated and blended modes. This simplifies the API and reduces redundancy since both modes apply FiLM modulation after activation in the same way. Changes: - Removed gating_activation_post_film_params parameter from _Layer, _LayerArray, and LayerArrayParams constructors - Removed _gating_activation_post_film member variable from _Layer - Updated _Layer::Process() to use _activation_post_film for gated mode - Updated all test files to use 7 FiLM parameters instead of 8 - Updated weight count in test_real_time_safe.cpp accordingly
Update WaveNet C++ code to handle secondary_activation as ActivationConfig instead of string for proper type safety. This enables support for complex activation types with parameters (e.g., PReLU, LeakyHardtanh). Changes: - Modify _Layer, _LayerArray, and LayerArrayParams to use typed ActivationConfig - Update Factory function to parse secondary_activation from JSON as ActivationConfig - Update all test files to use ActivationConfig for secondary activation parameters All tests pass successfully.
Fixed two bugs in _Layer constructor: - Conv1D was missing groups_input parameter (always defaulted to 1) - Conv1x1 _1x1 was passing groups_1x1 as bias parameter instead of groups These fixes enable proper grouped convolutions for reduced computation.
…ation FiLM. Failing.
- Fix incorrect parameter comments (lines 713-720): corrected parameter names to match actual Layer constructor - Fix misleading comment on activation_post_film weight calculation: clarify that FiLM is created with bottleneck as input_dim, shift doubles output channels - Remove 4 extra placeholder weights that were causing assertion failures - Apply same fixes to test_layer_post_activation_film_blended_realtime_safe
…on paths - Use Eigen::Ref in FiLM::Process and Conv1x1::process_ to accept block expressions without creating temporary matrices - Add pre-allocated buffers in GatingActivation and BlendingActivation to avoid allocating MatrixXf objects in processing loops
When head_1x1 was active with out_channels != bottleneck, _head_inputs and _head_rechannel were incorrectly sized using bottleneck instead of head_1x1.out_channels, causing an Eigen matrix dimension mismatch. Added _head_output_size member to _LayerArray that correctly computes the head output size (head_1x1.out_channels if active, else bottleneck). Updated weight generator to match.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
big exhale
Bugs fixed:
wavenet::Layerwhen head 1x1 is activeAdded
wavenet_a2_max.nam, which shows off all of the new features in v0.4.0.Resolves #197