Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
299 commits
Select commit Hold shift + click to select a range
21ba407
Made LUT parameterisable by width but not by size
J-u1i0 Mar 10, 2024
7e0bfe6
Fixed testbench and reduced register sizes of nr stage
J-u1i0 Mar 11, 2024
d7c247d
Updated the testbenches to import software module parts from a single…
J-u1i0 Mar 11, 2024
7ae18b9
All isqrt test passing
J-u1i0 Mar 11, 2024
8670894
Fixed assert stmt
J-u1i0 Mar 11, 2024
60a8a32
factored out lut parameter dict
dereklai1 Mar 12, 2024
0e52760
invsqrt working but group norm sometimes has 1 bit error, fix is WIP
dereklai1 Mar 12, 2024
d2670a0
wip: groupnorm2d still failing test, have no clue how it is expecting…
dereklai1 Mar 12, 2024
aa896a3
remove nettype none for vivado bug
Mar 12, 2024
88e434a
added tcl script for vivado
Mar 12, 2024
8619dda
changed all software quantized modules
dereklai1 Mar 14, 2024
54fc5f8
added an error threshold monitor, groupnorm2d still has very weird hi…
dereklai1 Mar 14, 2024
2dfbab0
updated error threshold monitor to support no check
dereklai1 Mar 14, 2024
781b688
major fix to fifo, groupnorm2d now passing
dereklai1 Mar 14, 2024
94f4123
add instance norm and fixed software emit verilog
dereklai1 Mar 14, 2024
366d54c
added batchnorm2d, groupnorm and instance norm quantizable modules in…
dereklai1 Mar 14, 2024
f5a71bc
removed fixed signed cast
dereklai1 Mar 15, 2024
4aff8e9
added lut common element
dereklai1 Mar 15, 2024
26150c1
removed local path
dereklai1 Mar 15, 2024
ad5ae52
integrated new lut into fixed isqrt
dereklai1 Mar 15, 2024
e0eef35
fixed streaming mon, added new lut isqrt into groupnorm 2d, added mor…
dereklai1 Mar 15, 2024
a9400c7
added unified norm testbench and removed old temporary inv sqrt
dereklai1 Mar 15, 2024
744d69f
updated emit verilog tb, top level norm still failing
dereklai1 Mar 15, 2024
1eba479
fixed isqrt linting error
dereklai1 Mar 15, 2024
d069332
removed unused params from tb
dereklai1 Mar 15, 2024
609963a
pipelined isqrt module and integrated into group norm 2d
dereklai1 Mar 15, 2024
36886f0
added new rms norm extension layers, hardware not passing tb
dereklai1 Mar 18, 2024
524073a
fixed rms norm implementation and added tests for hw
dereklai1 Mar 19, 2024
3183671
added rms norm to software stack
dereklai1 Mar 19, 2024
69d1b11
refactored rms_norm and test_emit_verilog_norm test
dereklai1 Mar 19, 2024
dbad6f2
commented out rms_norm circular dependency issue not fixed
dereklai1 Mar 19, 2024
d042424
normtb works for rms
dereklai1 Mar 19, 2024
5334943
removed online layer norm
dereklai1 Mar 19, 2024
aeb1bfb
moved inverse sqrt C++ fixed model
dereklai1 Mar 19, 2024
6fe7f42
fixed groupnorm net in test_emit_verilog_norm
dereklai1 Mar 19, 2024
31a60c1
refactored test_emit_verilog_norm and readjusted test times
dereklai1 Mar 19, 2024
3c199b4
docstring
dereklai1 Mar 19, 2024
ef48c93
add memoryfile and top for vivado synth
dereklai1 Mar 19, 2024
f67e60b
vivado script
Mar 19, 2024
bcd2382
fixed top level params & ports
Mar 19, 2024
6b4d93b
Batch norm 2d working
J-u1i0 Mar 19, 2024
8f7a796
Merge branch 'layernorm_sqrt' of github.com:dereklai1/mase into layer…
J-u1i0 Mar 19, 2024
b1f00e1
added comments
dereklai1 Mar 19, 2024
1ae1eaf
added u250 constraints file and updated tcl
Mar 19, 2024
8a956a7
stripped xdc file
dereklai1 Mar 19, 2024
82de30c
docstrings
dereklai1 Mar 19, 2024
c8e71d9
Batch norm hook up?
J-u1i0 Mar 19, 2024
a5185f2
Merge branch 'layernorm_sqrt' of github.com:dereklai1/mase into layer…
J-u1i0 Mar 19, 2024
0a18c29
integrated batchnorm into norm.sv all tests passing
dereklai1 Mar 19, 2024
528ba4c
pipelined batchnorm module
dereklai1 Mar 20, 2024
6030acc
Merge branch 'main' into group_17_norm
dereklai1 Mar 20, 2024
503f72d
added docs for normalization layers
dereklai1 Mar 21, 2024
9c0471d
fixed bugged channel selector
dereklai1 Mar 23, 2024
41eef4b
added per-element scale to rms norm
dereklai1 Mar 24, 2024
8f21dac
fixed rms norm function, norm_tb rms not passing
dereklai1 Mar 24, 2024
6a85bad
error analysis code
dereklai1 Mar 25, 2024
51b2d61
added fifo
dereklai1 Mar 26, 2024
79b8ee2
switched to non-project vivado flow and 333mhz clk
dereklai1 Mar 26, 2024
e6b7490
removed variance clamp from groupnorm
dereklai1 Mar 27, 2024
da9d6f6
fix comma
dereklai1 Mar 27, 2024
37a1908
added isqrt fix and inverse NUM_VALUES fix to rms norm
dereklai1 Mar 27, 2024
b0c1857
moved around wires for vivado
dereklai1 Mar 27, 2024
1068e4d
isqrt pipelineing
dereklai1 Mar 27, 2024
66aaedf
cleaned up nr stage
dereklai1 Mar 27, 2024
7572155
vivado scripts
dereklai1 Mar 28, 2024
eebd4ab
fix batchnorm
dereklai1 Mar 28, 2024
27647d5
comment out batchnorm
dereklai1 Mar 28, 2024
243b815
pipelined NR stage more
dereklai1 Mar 28, 2024
4833fae
fix nr stage bug
dereklai1 Mar 28, 2024
b1ae560
more aggressive clk frequency
dereklai1 Mar 28, 2024
5308d09
Merge branch 'main' into group_17_norm
dereklai1 Mar 28, 2024
0d4124d
added two contraint files to find fmax
dereklai1 Mar 28, 2024
2cbc083
fixed timing constraints
dereklai1 Mar 29, 2024
ec4d76a
fixed up tests for submission
dereklai1 Mar 29, 2024
59bb74b
sv linting
dereklai1 Mar 29, 2024
714c338
python linting
dereklai1 Mar 29, 2024
81a9629
more linting
dereklai1 Mar 29, 2024
23abde2
isqrt CI
dereklai1 Mar 29, 2024
a99eee2
fixed tests
dereklai1 Mar 29, 2024
f8ad341
fix test_emit_verilog_norm
dereklai1 Mar 29, 2024
16cd689
removed cpp model duplicate
dereklai1 Mar 29, 2024
d3c4f01
fixed software tests
dereklai1 Mar 29, 2024
31dbc2b
reformat for pytest
dereklai1 Mar 29, 2024
a1c955c
linting
dereklai1 Mar 29, 2024
61cd7bf
sv linting
dereklai1 Mar 29, 2024
bc2569d
python linting
dereklai1 Mar 29, 2024
4933667
linting
dereklai1 Mar 29, 2024
29d8259
docstr
dereklai1 Mar 29, 2024
9d9866f
removed matmul
dereklai1 Mar 30, 2024
3ea417b
Trigger Build
dereklai1 Mar 30, 2024
121c93c
group norm CI
dereklai1 Mar 30, 2024
fa5941d
ADLS: Hardware Normalization (Group 17) (#65)
dereklai1 Apr 3, 2024
0353acb
Updated CI rules to apply on more branches
Apr 3, 2024
7bdd573
group 8 (#123)
Apr 3, 2024
bfd3a04
Fix software train runner RunnerBasicTrain (#90)
btarifi10 Apr 3, 2024
1744136
Merge branch 'group_17_norm' into gqa
dereklai1 Apr 9, 2024
7f6500e
WIP: pow2 rtl for softmax norm
dereklai1 Apr 9, 2024
2595e31
first working draft of pow2
dereklai1 Apr 11, 2024
28b91bd
added comparator tree
dereklai1 Apr 11, 2024
467197f
more tests
dereklai1 Apr 11, 2024
2f371ba
add common metadata for BERT masegraph generated from ONNX backend
pgimenes Apr 11, 2024
00176d1
added comparator accumulator
dereklai1 Apr 12, 2024
c67b8d8
added first section of softermax
dereklai1 Apr 12, 2024
3b9878b
formalize onnx ir and export pass
pgimenes Apr 12, 2024
8199a49
added splitn (untested) and reworked range reduction + lpw_pow2
dereklai1 Apr 15, 2024
2e0f393
enable creating a masegraph directly from fx graphmodule
pgimenes Apr 16, 2024
facb123
x
dereklai1 Apr 16, 2024
f58d9ba
bert model from codegen working, but generating different output
pgimenes Apr 16, 2024
64b64c4
temporarily disable extended model testing
pgimenes Apr 16, 2024
e47922d
support for more ops, refactor attr mapping as dict, BERT works, need…
pgimenes Apr 17, 2024
d3d1c46
small fixes
pgimenes Apr 17, 2024
7db2b89
only test on bert
pgimenes Apr 17, 2024
a181dbe
fix tests and docs
pgimenes Apr 18, 2024
50ed11e
Merge branch 'main' of https://github.com/DeepWok/mase into onnx
pgimenes Apr 18, 2024
28b9d40
Merge branch 'main-adls-2324' of https://github.com/DeepWok/mase into…
pgimenes Apr 18, 2024
b6d8163
tidy up documentation
pgimenes Apr 18, 2024
683274a
matmul version for fixed linear
pgimenes Apr 19, 2024
d40410b
added range of tests for lpw recip
dereklai1 Apr 22, 2024
1fbfd52
ADLS Group 7 (#106)
pgimenes Apr 23, 2024
3060f4b
ADLS Group 17 (#118)
pgimenes Apr 26, 2024
cd058dd
ADLS Group 13 (#103)
pgimenes Apr 27, 2024
34dacf4
ADLS Group 12 (#127)
pgimenes Apr 27, 2024
c57fc0f
started raise granularity transform pass
pgimenes Apr 27, 2024
a256161
adder tree changes to fix synthesis, Makefile generalization
pgimenes Apr 27, 2024
e9d58a5
Merge branch 'main' of https://github.com/DeepWok/mase into linear-dump
pgimenes Apr 27, 2024
97e77fe
fixed reciprocal width
dereklai1 Apr 29, 2024
5dead8f
unsigned cast module WIP
dereklai1 Apr 29, 2024
7a3ae72
fix runner
dereklai1 Apr 29, 2024
c81ad8f
fix software model
dereklai1 Apr 29, 2024
7dc32c4
lpw reciprocal passing all tests
dereklai1 Apr 29, 2024
8701dfb
all tests passing for pow2
dereklai1 Apr 29, 2024
b914cd4
fixed broken test for local window
dereklai1 Apr 30, 2024
f8a4ebb
fixed high width problem in lpw recip and pow2
dereklai1 Apr 30, 2024
9a727de
temp softermax
dereklai1 May 1, 2024
3a1523b
linear layer add bias and fix testbench, data consistency passing
pgimenes May 2, 2024
cd12976
more tests for fixed linear
pgimenes May 2, 2024
f1ca9d9
fixed self attention hardware module and BertAttentionInteger quantiz…
pgimenes May 2, 2024
55eec25
Merge remote-tracking branch 'origin/main-adls-2324' into galapagos-m…
pgimenes May 2, 2024
ced9695
minor changes
dereklai1 May 2, 2024
1f0e3b9
softermax top level testbench
pgimenes May 3, 2024
4d49a4e
started bert self attention implementation
pgimenes May 3, 2024
bdc370c
cherry pick derek's softermax
pgimenes May 3, 2024
0633b25
fixed comparator accum bug & global norm testbench
dereklai1 May 3, 2024
de41b62
2d softermax
pgimenes May 4, 2024
eb819e3
attention head control flow passing, moved attention model to chop.nn…
pgimenes May 4, 2024
4dde0b1
linting fixes
pgimenes May 4, 2024
3ab0aa9
batched input block splitting issue, fix formatting
pgimenes May 5, 2024
ae5ada8
fifo to latency match QK^T path. self attention control flow passing
pgimenes May 5, 2024
2fa6d21
added assert to repeat_circ_buffer
dereklai1 May 7, 2024
538f3f9
added single element repeater
dereklai1 May 7, 2024
3972cad
replaced repeat_circular_buffer with single_element_repeat to fix thr…
dereklai1 May 7, 2024
09a29b5
Merge remote-tracking branch 'origin/pedro-softermax' into gqa
dereklai1 May 7, 2024
641c4c9
verilator obj_dir gitignore
dereklai1 May 7, 2024
1f9a978
fix lpw pow2 and recip
dereklai1 May 7, 2024
b85a729
replaced circ buffer instances and fix softermax tb
dereklai1 May 7, 2024
5e121ed
common metadata and hardware metadata for patched bert using HF tracer
pgimenes May 7, 2024
1573327
raise granularity unfinished
pgimenes May 7, 2024
a97d141
fixed self attention needs softermax fix
pgimenes May 8, 2024
6788b1b
Merge remote-tracking branch 'origin/onnx-bert' into galapagos-mnist
pgimenes May 8, 2024
f974516
first version of hardware emitted
pgimenes May 8, 2024
8e3d6e2
fixed softermax bug and added more tests, all passing locally
dereklai1 May 9, 2024
44daee9
added first draft of fixed_attention_head
dereklai1 May 9, 2024
be00052
removed dead code
dereklai1 May 9, 2024
4071345
ready for merge
dereklai1 May 9, 2024
6219cba
moved self attention files from feature/bert
dereklai1 May 9, 2024
45eb863
added random tests
dereklai1 May 9, 2024
84d429e
fixes to linear
pgimenes May 9, 2024
b6cadcd
first draft of gqa head, not tested
dereklai1 May 10, 2024
fb21d3f
update emit tb for new components
pgimenes May 10, 2024
e45335a
Merge remote-tracking branch 'origin/mlp_example' into feature/bert
pgimenes May 10, 2024
20c4d58
added softermax changes & fixed dimension problem
dereklai1 May 10, 2024
f040576
fixed_gqa_head passing basic data tests, there is a throughput issue
dereklai1 May 10, 2024
7b5672a
added bunch of tests to gqa head
dereklai1 May 10, 2024
c13641f
patch FIFO to fix hold violations
pgimenes May 11, 2024
2a48fbf
first draft at making matmul/ synthesizable
dereklai1 May 12, 2024
637f0a5
temp
dereklai1 May 12, 2024
a5f7aae
removed comments
dereklai1 May 12, 2024
56009e4
replace fifo with skid buffer when SIZE=1
dereklai1 May 12, 2024
c97ae97
matmul and simple matmul passing
dereklai1 May 12, 2024
1deb66e
matrix stream transpose test passing
dereklai1 May 12, 2024
51a58c0
Merge remote-tracking branch 'derek/gqa' into feature/bert
pgimenes May 12, 2024
4da0b80
linting
pgimenes May 12, 2024
4296922
hackaround streaming driver to get linear tb working
dereklai1 May 12, 2024
4acd722
reduce size
pgimenes May 12, 2024
290bc03
first draft in fixing adder tree
dereklai1 May 12, 2024
6d45790
Merge remote-tracking branch 'origin/feature/bert' into matmul-synth
dereklai1 May 12, 2024
d170435
revert fix to fixed adder tree
pgimenes May 12, 2024
a27bb82
fixed adder tree for synthesis
dereklai1 May 12, 2024
ff8d30e
minor fixes
dereklai1 May 12, 2024
2b46724
linting
pgimenes May 12, 2024
91261b9
Merge remote-tracking branch 'derek/matmul-synth' into feature/bert
pgimenes May 12, 2024
a600f69
synthesis tests
pgimenes May 13, 2024
555abf6
fixed matmul bug and refactored test
dereklai1 May 13, 2024
2acf769
Merge remote-tracking branch 'derek/matmul-synth' into feature/bert
pgimenes May 13, 2024
3c40cc2
pass to emit vivado project and package IP
pgimenes May 15, 2024
ba8c389
Merge remote-tracking branch 'origin/main' into feature/bert
pgimenes May 15, 2024
ff92236
move files around to make merge easier
pgimenes May 15, 2024
c90c163
Merge remote-tracking branch 'origin/main' into feature/bert
pgimenes May 15, 2024
dc353f1
remove duplicated files and leftover merge guard
pgimenes May 16, 2024
4703b3a
test for emit bert
pgimenes May 16, 2024
83883fa
formatting
pgimenes May 16, 2024
b9b3169
sw formatting
pgimenes May 16, 2024
0aef5ec
fixed for bert emit. emit bram needs fixing
pgimenes May 16, 2024
ba18a85
temporary solution to split nodes for bert residual connection. fixed…
pgimenes May 20, 2024
ad7f4d4
split_n comb loop fix + struct refactoring for verilator
pgimenes May 21, 2024
2863863
smaller config to speed up build
pgimenes May 21, 2024
55014ad
fixes
pgimenes May 21, 2024
d920b9e
metadata update explanation and unpacked fifo in buffer
pgimenes May 21, 2024
7b29d32
buffer fix
pgimenes May 21, 2024
7a66bfe
evert FIFO version and temporarily disable bias
pgimenes May 21, 2024
b4b2092
replace debug function, replace GELU implementation
pgimenes May 22, 2024
f3420ad
don't skip build
pgimenes May 22, 2024
27e4add
pre-build gelu lut
pgimenes May 22, 2024
45c973a
enable bram again
pgimenes May 22, 2024
dc971fa
improve graph reporting
pgimenes May 22, 2024
d363fc2
linear matmul weights dimension parameter fix
pgimenes May 22, 2024
468958a
finish cleanly when all monitors are drained
pgimenes May 22, 2024
8797e58
increment trace depth
pgimenes May 22, 2024
83654ef
report build and test times for simulate action
pgimenes May 22, 2024
3f5a8fe
fix linear bias
pgimenes May 23, 2024
c3b130d
pass linear synth check
pgimenes May 23, 2024
d5f6c19
update linear tb
pgimenes May 23, 2024
419471a
emit verilog linear fixes (for galapagos)
pgimenes May 23, 2024
9577f09
match sequence length with parallelism in emit bert smoke test
pgimenes May 23, 2024
0bc0d74
mount board files in make shell
pgimenes May 23, 2024
c9548f8
Merge branch 'feature/bert' of https://github.com/DeepWok/mase into f…
pgimenes May 23, 2024
9ea719b
fix reg_slicer related stuff when calling python src/mase_components/…
Aaron-Zhao123 May 24, 2024
0c6f92e
Merge remote-tracking branch 'refs/remotes/origin/feature/bert' into …
Aaron-Zhao123 May 24, 2024
7e08de4
now passed python3 test/passes/graph/transforms/verilog/test_emit_ver…
Aaron-Zhao123 May 24, 2024
308aca3
nas bert
pgimenes May 29, 2024
b8c2ff3
Merge branch 'feature/bert' of https://github.com/DeepWok/mase into f…
pgimenes May 29, 2024
0ac62fd
[NAS-BERT] layerwise search space working
pgimenes May 29, 2024
17d4a24
[NAS-BERT] include num heads/hidden dim in search space, plot pareto
pgimenes May 30, 2024
28b5b4e
compatibility changes for questasim
pgimenes Jun 3, 2024
911cfe7
comparator fix to pass larger config
pgimenes Jun 4, 2024
340f9b7
fix mount for beholder and register old gelu removal
pgimenes Jun 4, 2024
b5bdad3
initial commit
Jun 4, 2024
8158db1
make simulator choice explicit
pgimenes Jun 4, 2024
568a822
initialize llama, mistral
pgimenes Jun 6, 2024
9db2122
auto pipeline for emit verilog
pgimenes Jun 6, 2024
bcb9112
changes for emitting llama
pgimenes Jun 6, 2024
b67d712
cvxpy bug for resharding cost term in intra operator pass
pgimenes Jun 6, 2024
82b706e
update resharding cost model and ILP constraints
pgimenes Jun 11, 2024
e1cd29a
some refactoring
pgimenes Jun 11, 2024
b3ecaac
attach to runtime
pgimenes Jun 11, 2024
df4a02b
handle resharding between nodes, improve logging
pgimenes Jun 12, 2024
74c2712
make sharding decision dict based instead of positional
pgimenes Jun 12, 2024
6d2a9b9
Merge branch 'research/alpa-light' into feature/bert
pgimenes Jun 12, 2024
63b633b
run autosharding on huggingface bert
pgimenes Jun 13, 2024
661aec8
emit verilog for bert doc
pgimenes Jul 12, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,7 @@ coverage.xml

# Hardware tests
build/
obj_dir/

# Translations
*.mo
Expand Down Expand Up @@ -143,6 +144,16 @@ venv.bak/
# Data files for hardware simulation
*.dat

# Vivado
*.xpr
*.jou
*.cache
*.runs
*.srcs
*.hw
vivado_pid*
synth/

# Generated SV LUT files
*_lut.sv

Expand Down
14 changes: 10 additions & 4 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,15 @@ VIVADO_AVAILABLE := $(shell command -v vivado 2> /dev/null)
ifeq ($(GPU_AVAILABLE),)
PLATFORM := cpu
else
PLATFORM := cuda
PLATFORM := gpu
endif

# * Mount Vivado HLS path only if Vivado is available (to avoid path not found errors)
# Include shared folder containing board files etc
ifeq ($(VIVADO_AVAILABLE),)
DOCKER_RUN_EXTRA_ARGS=
else
DOCKER_RUN_EXTRA_ARGS=-v $(vhls):$(vhls)
DOCKER_RUN_EXTRA_ARGS= -v /mnt/applications/:/mnt/applications -v $(vhls):$(vhls) -v /$(USER_PREFIX)/$(shell whoami)/shared:/root/shared
endif

# * Set docker image according to local flag
Expand All @@ -42,6 +43,11 @@ hw_test_dir = src/mase_components/

NUM_WORKERS ?= 1

sw_test_dir = machop/test/
hw_test_dir = machop/mase_components/

NUM_WORKERS ?= 1

# Make sure the repo is up to date
sync:
git submodule sync
Expand Down Expand Up @@ -70,8 +76,8 @@ shell:
-w /workspace \
-v /$(USER_PREFIX)/$(shell whoami)/.gitconfig:/root/.gitconfig \
-v /$(USER_PREFIX)/$(shell whoami)/.ssh:/root/.ssh \
-v /$(USER_PREFIX)/$(shell whoami)/.mase:/root/.mase:z \
-v $(shell pwd):/workspace:z \
-v /$(USER_PREFIX)/$(shell whoami)/.mase:/root/.mase \
-v $(shell pwd):/workspace \
$(DOCKER_RUN_EXTRA_ARGS) \
$(img) /bin/bash

Expand Down
6 changes: 6 additions & 0 deletions docs/source/modules/documentation/tutorials.rst
Original file line number Diff line number Diff line change
@@ -1,6 +1,12 @@
Tutorials
=============================

.. toctree::
:maxdepth: 1
:caption: Common Usecases

tutorials/common/bert_emit

.. toctree::
:maxdepth: 1
:caption: The Command Line System with Machop
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# Deploy a BERT model on an FPGA

2 changes: 1 addition & 1 deletion docs/source/modules/hardware/arithmetic/mac.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
# Multiply-Accumulate (MAC) Unit

![MAC](https://raw.githubusercontent.com/DeepWok/mase/main/docs/source/imgs/hardware/mac.png)
![MAC](https://raw.githubusercontent.com/DeepWok/mase/main/docs/source/imgs/hardware/mac.png)
Loading