76c3244cc5
* initial commit for command-R * update mlp, layernorm, lm_head and model args * add custom layernorm * add default to tie_word_embeddings * add layernorm weight type and refactor * update layernorm (bias conditional) in model/layers * fix layer norm use traditional rope * add test --------- Co-authored-by: Awni Hannun <awni@apple.com> |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| base.py | ||
| cohere.py | ||
| gemma.py | ||
| layers.py | ||
| llama.py | ||
| mixtral.py | ||
| olmo.py | ||
| phi.py | ||
| phixtral.py | ||
| plamo.py | ||
| qwen.py | ||
| qwen2.py | ||
| stablelm.py | ||
| starcoder2.py | ||