ReMe/reme/steps/cookbook/dingtalk/send.py
jinliyl 46adb5ae1e
Some checks failed
Pre-commit / run (ubuntu-latest) (push) Has been cancelled
Tests ReMe / Unit Tests - py3.11 (push) Has been cancelled
Tests ReMe / Unit Tests - py3.12 (push) Has been cancelled
Tests ReMe / Unit Tests - py3.13 (push) Has been cancelled
Windows Smoke / CLI smoke - py3.11 (push) Has been cancelled
feat: add daily paper cookbook and DingTalk agent integration (#385)
* feat(daily-paper): add daily paper cookbook workflow with schema and tests

- Introduce daily paper schema types (DailyBriefOutput, PaperInfo, PaperNoteOutput, etc.)
- Create daily paper cookbook module with analyze, collect, digest, rank, and select steps
- Add cookbook entry point and integrate into main steps module
- Replace job config export with daily brief output in schema exports
- Add comprehensive unit tests covering pipeline, filtering, and output generation
- Update dependencies including openai-codex and pypdf packages
- Configure standalone daily paper cron job with proper scheduling and routing

* test(daily_paper): update tests to use Claude Code wrapper exclusively

- Add test to verify web search is disallowed by default in Claude Code
- Update imports to include DailyBriefOutput, PaperNoteOutput, and PaperSelection schemas
- Change test name from standalone_config_has_backend_split to reflect Claude Code only usage
- Remove default agent wrapper and configure all steps to use Claude Code wrapper
- Rename select_wrapper to cc_wrapper for clarity and consistency
- Remove duplicate Claude Code wrapper initialization
- Update test assertions to verify output schema usage matches expected sequence
- Remove unused as_llm component from standalone configuration test

* refactor(agent-wrapper): simplify skill resolution logic across all wrappers

- Replace duplicate skill resolution code with centralized _resolve_project_skills method
- Add project_path property with configurable relative path resolution
- Introduce proper validation for skill names and directory existence
- Change Codex wrapper to use project_path instead of workspace_path for skills
- Add SKILL.md requirement validation for project skills
- Remove redundant skill processing logic from individual wrappers

* feat(daily_paper): add daily paper workflow with PDF analysis and brief generation

- Implement shared state management and file helpers for daily-paper steps
- Add PDF download and text extraction capabilities with arXiv integration
- Create paper collection step with Hugging Face weekly/monthly rankings
- Build ranking system using reciprocal-rank fusion with memory keyword scoring
- Add Claude Code integration for paper analysis and detailed note generation
- Implement digest step to create final five-minute brief from detailed notes
- Add configuration for standalone daily cookbook application with cron scheduling
- Create typed schema for paper information, selection, and output formats
- Add atomic file writing with temporary file safety mechanisms
- Implement exclusion logic for previously recommended papers and daily filters

* feat(daily_paper): add DingTalk notification integration and enhance logging

- Integrate DingTalk markdown send step to notify groups about daily paper briefs
- Add comprehensive logging throughout daily paper workflow including start/finish events
- Update daily paper analysis prompt to include code repository context requirement
- Configure DingTalk notification in daily_cookbook.yaml with app credentials
- Add dingtalk-stream dependency for proactive message API integration
- Enhance daily paper README with DingTalk notification section and updated flow chart
- Implement detailed logging for each step including paper processing and agent calls
- Add test coverage for DingTalk markdown sending functionality and configuration
- Update pre-commit config to exclude skills directory from checks
- Add .claude/skills to gitignore for local development environment

* refactor(dingtalk): move dingtalk_stream import to local scope and improve code safety

- Moved global dingtalk_stream import to local scope in send.py to avoid eager loading
- Added dynamic import with error handling for optional dependency cases
- Updated test suite to verify lazy loading behavior works correctly
- Fixed markdown title generation by using safe variable naming in wait.py
- Enhanced test coverage for arxiv PDF download caching functionality
- Updated application context initialization with proper resource directory configuration
- Modified paper metadata to include source PDF path reference in output files

* refactor(daily_paper): remove manifest system and store selection metadata in digest files

- Remove JSON manifest creation and storage functionality
- Store selection data directly in digest file frontmatter instead of separate manifest files
- Add load_saved_selection method to rebuild selection from digest and paper-note metadata
- Update README documentation to reflect new cookbook workflow architecture
- Modify test cases to verify selection metadata in digest files instead of manifest JSON
- Remove unused json import from multiple daily paper modules
- Integrate PaperSelection schema for proper data validation in stored metadata

* docs(daily_paper): add bilingual cookbook guides
2026-07-22 19:17:01 +08:00

123 lines
5.5 KiB
Python

"""Send a workspace Markdown file to DingTalk group conversations."""
import asyncio
import json
import aiofiles
import frontmatter
import httpx
from ....components import R
from ...base_step import BaseStep
from ...file_io._path import gate_md, resolve_path
_GROUP_SEND_URL = "https://api.dingtalk.com/v1.0/robot/groupMessages/send"
def _conversation_ids(value: str) -> list[str]:
return [item.strip() for item in value.split(",") if item.strip()]
@R.register("dingtalk_markdown_send_step")
class DingTalkMarkdownSendStep(BaseStep):
"""Send one Markdown document serially to configured DingTalk groups."""
def __init__(
self,
app_key: str = "",
app_secret: str = "",
robot_code: str = "",
conversation_ids: str = "",
title: str = "",
timeout: float = 15.0,
**kwargs,
):
super().__init__(**kwargs)
self.app_key = app_key
self.app_secret = app_secret
self.robot_code = robot_code
self.conversation_ids = _conversation_ids(conversation_ids)
self.title = title
self.timeout = timeout
async def execute(self):
assert self.context is not None
recipients = self.conversation_ids
self.context.response.metadata["dingtalk_configured_count"] = len(recipients)
self.context.response.metadata["dingtalk_sent_count"] = 0
if not recipients:
self.logger.info(f"[{self.name}] skipped DingTalk Markdown delivery: no conversation IDs")
return self.context.response
if not all((self.app_key, self.app_secret, self.robot_code)):
raise RuntimeError("DingTalk Markdown delivery requires app_key, app_secret, and robot_code")
raw_path = str(self.context.get("markdown_path") or "")
if not raw_path:
if self.context.response.metadata.get("skipped"):
self.logger.info(f"[{self.name}] skipped DingTalk Markdown delivery: no markdown path")
return self.context.response
raise RuntimeError("DingTalk Markdown delivery requires markdown_path")
target, error = resolve_path(self.workspace_path, raw_path)
if error:
raise ValueError(f"Invalid DingTalk Markdown path: {error}")
assert target is not None
target, is_markdown = gate_md(target)
if not is_markdown:
raise ValueError("DingTalk Markdown delivery requires a .md file")
if not target.is_file():
raise FileNotFoundError(f"DingTalk Markdown file does not exist: {raw_path}")
async with aiofiles.open(target, encoding="utf-8") as stream:
document = frontmatter.loads(await stream.read())
markdown = document.content.strip()
if not markdown:
raise ValueError(f"DingTalk Markdown file is empty: {raw_path}")
import dingtalk_stream # pylint: disable=import-outside-toplevel
title = self.title or str(document.metadata.get("name") or target.stem)
token_client = dingtalk_stream.DingTalkStreamClient(
dingtalk_stream.Credential(self.app_key, self.app_secret),
)
access_token = await asyncio.to_thread(token_client.get_access_token)
if not access_token:
raise RuntimeError("Failed to obtain DingTalk access token")
self.logger.info(
f"[{self.name}] sending DingTalk Markdown path={raw_path} recipients={len(recipients)} "
f"chars={len(markdown)} timeout={self.timeout:.1f}s",
)
failures: list[str] = []
headers = {"x-acs-dingtalk-access-token": access_token, "User-Agent": "ReMe DingTalk notifier"}
transport = httpx.AsyncHTTPTransport(local_address="0.0.0.0")
async with httpx.AsyncClient(timeout=self.timeout, headers=headers, transport=transport) as client:
for index, conversation_id in enumerate(recipients, start=1):
payload = {
"robotCode": self.robot_code,
"openConversationId": conversation_id,
"msgKey": "sampleMarkdown",
"msgParam": json.dumps({"title": title, "text": markdown}, ensure_ascii=False),
}
try:
response = await client.post(_GROUP_SEND_URL, json=payload)
response.raise_for_status()
result = response.json()
if not isinstance(result, dict) or not result.get("processQueryKey"):
raise ValueError("missing processQueryKey")
except (httpx.HTTPError, ValueError) as exc:
failures.append(f"recipient {index}: {type(exc).__name__}")
self.logger.warning(
f"[{self.name}] DingTalk delivery failed recipient={index}/{len(recipients)} "
f"error_type={type(exc).__name__}",
)
continue
self.context.response.metadata["dingtalk_sent_count"] += 1
self.logger.info(f"[{self.name}] delivered DingTalk Markdown recipient={index}/{len(recipients)}")
sent_count = self.context.response.metadata["dingtalk_sent_count"]
if failures:
self.context.response.metadata["dingtalk_delivery_errors"] = failures
raise RuntimeError(f"DingTalk Markdown delivery failed for {len(failures)} of {len(recipients)} recipients")
self.logger.info(f"[{self.name}] DingTalk Markdown delivery complete sent={sent_count} total={len(recipients)}")
return self.context.response