* add jax profiler server

* update jetstream

* add interleave multiple host with ray readme

* add interleave multiple host with ray readme

* Fix

* Format

* add diaggregated server with ray support

* add run_server wity ray

* format

* support converting hf checkpoint

add doc

* Add support for Llama3-70b

* Fix unit tests

* assert model_name is one of llama-2 or llama-3 for weight sharding

* Fix lint

* Revert separate shardings for llama-2 and llama-3

* Fix lint

* Initial Mixtral enablement.

* Adds the mistral tokenizer model.

* Updates the convert checkpoint file to handle mistral model.

* Renames the typo of the model name.

* Fixing checkpoing loading. Still has some issue. Push to debug.

* Running on CPU working, temporarily disable the generate jit to see it's moving. But the outputs doesn't make sense yet because weights are not loaded yet.

* Fix checkpoint loading issue. Right now loading from the gpt-fast converter with qkv fusion.

* Fix the ckpt conversion script for mistral model. Fix the freqs_cis for loading pth file.

* Add quantized layer for moe

quantization

* Add the huggingface download script. Improved the convert checkpoints logging.

* Clean up and fix lint errors.

* Missing cleanups.

* Add instructions for Mixtral.

* Renames everything from mistral to mixtral.

* Fix more lint errors.

* Removes the unnecessary checkpoint name mapping from the original Mixtral checkpoints.

* Fix the model calling arg sequence; Fix the checkpoint convert script.

---------

Co-authored-by: Han Qi 

…ayers (#105)

* add activation quant support

* pyink

* fix dtype

* uncomment prompts

* try fix test

add debug print to debug

remove print, add bias to asym quant tests

lint

* add comment

update gemma convert

add lock for prefill and generate to prevent starvation

* make sure GPU works