mirror of
https://github.com/agentscope-ai/ReMe.git
synced 2026-08-28 05:25:04 +00:00
b9da8d2a7f
- Add new ToolCallResult and ToolMemory schemas for tracking tool executions - Implement tool memory retrieval and summarization flows in default config - Register new parse_tool_call_result_op for processing tool call results - Extend memory conversion logic to support tool memory type - Add test cases for tool memory serialization and deserialization - Include token counting utilities for text processing tasks
6 lines
No EOL
163 B
Python
6 lines
No EOL
163 B
Python
import tiktoken
|
|
enc = tiktoken.get_encoding("o200k_base")
|
|
|
|
# r = enc.encode("我爱吃西瓜，你说啥")
|
|
r = enc.encode("hello world aaaaaaaaaaaa")
|
|
print(len(r)) |