Thank's for this post,
I tried it for a cpu config under Linux.
Had 2 problems :
- in the first python prog under "Run the model using a Python Script and switch command for model selection",
no such section can be found
if __name__ == "__main__":
I took the one from the 2nd python program which sounds good.
Then, on the two programs, I get the following error :
Loading model...
Traceback (most recent call last):
File "phi_chat.py", line 86, in <module>
main(args)
File "phi_chat.py", line 18, in main
model = og.Model(f'{args.model}')
^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Error opening cpu_and_mobile/cpu-int4-rtn-block-32-acc-level-4/genai_config.json
If I do a find command, I get no genai_config.json file but only those :
added_tokens.json.lock
added_tokens.json.metadata
config.json.lock
config.json.metadata
configuration_phi3.py.lock
configuration_phi3.py.metadata
genai_config.json.lock
genai_config.json.metadata
phi3-mini-4k-instruct-cpu-int4-rtn-block-32-acc-level-4.onnx.data.lock
phi3-mini-4k-instruct-cpu-int4-rtn-block-32-acc-level-4.onnx.data.metadata
phi3-mini-4k-instruct-cpu-int4-rtn-block-32-acc-level-4.onnx.lock
phi3-mini-4k-instruct-cpu-int4-rtn-block-32-acc-level-4.onnx.metadata
special_tokens_map.json.lock
special_tokens_map.json.metadata
tokenizer_config.json.lock
tokenizer_config.json.metadata
tokenizer.json.lock
tokenizer.json.metadata
tokenizer.model.lock
tokenizer.model.metadata
What am'I missing ?
thank's