Merge pull request #230 from mistralai/add_layer_wise_rotated_cache
Add per-layer sliding window
No related branches found
No related tags found
Showing
- src/mistral_inference/args.py 7 additions, 1 deletionsrc/mistral_inference/args.py
- src/mistral_inference/cache.py 91 additions, 29 deletionssrc/mistral_inference/cache.py
- src/mistral_inference/generate.py 1 addition, 0 deletionssrc/mistral_inference/generate.py
- src/mistral_inference/transformer.py 17 additions, 6 deletionssrc/mistral_inference/transformer.py
Loading
Please register or sign in to comment