Initial commit: 知识管理平台

主要功能:
- 多用户管理系统
- 浏览器自动化(Playwright)
- 任务编排和执行
- Docker容器化部署
- 数据持久化和日志管理

技术栈:
- Flask 3.0.0
- Playwright 1.40.0
- SQLite with connection pooling
- Docker + Docker Compose

部署说明详见README.md
This commit is contained in:
Yu Yon
2025-11-16 19:03:07 +08:00
commit 0fd7137cea
23 changed files with 12061 additions and 0 deletions

44
.gitignore vendored Normal file
View File

@@ -0,0 +1,44 @@
# 浏览器二进制文件
playwright/
ms-playwright/
# 数据库文件(敏感数据)
data/*.db
data/*.db-shm
data/*.db-wal
data/secret_key.txt
# 日志文件
logs/
*.log
# 截图文件
截图/
# Python缓存
__pycache__/
*.py[cod]
*.class
*.so
.Python
env/
venv/
ENV/
# Docker volumes
volumes/
# IDE
.vscode/
.idea/
*.swp
*.swo
# 系统文件
.DS_Store
Thumbs.db
# 临时文件
*.tmp
*.bak
*.backup

46
Dockerfile Normal file
View File

@@ -0,0 +1,46 @@
# 使用国内镜像源加速
FROM mcr.microsoft.com/playwright/python:v1.40.0-jammy
# 设置工作目录
WORKDIR /app
# 设置环境变量
ENV PYTHONUNBUFFERED=1
ENV PLAYWRIGHT_BROWSERS_PATH=/ms-playwright
ENV TZ=Asia/Shanghai
# 配置 pip 使用国内镜像源
RUN pip config set global.index-url https://mirrors.aliyun.com/pypi/simple/ && pip config set install.trusted-host mirrors.aliyun.com
# 复制依赖文件
COPY requirements.txt .
# 安装Python依赖
RUN pip install --no-cache-dir -r requirements.txt
# 复制应用程序文件
COPY app.py .
COPY database.py .
COPY db_pool.py .
COPY playwright_automation.py .
COPY browser_installer.py .
COPY password_utils.py .
# 复制新的优化模块
COPY app_config.py .
COPY app_logger.py .
COPY app_security.py .
COPY app_state.py .
COPY app_utils.py .
COPY templates/ ./templates/
COPY static/ ./static/
# 创建必要的目录
RUN mkdir -p data logs 截图
# 暴露端口
EXPOSE 5000
# 启动命令
CMD ["python", "app.py"]

695
README.md Normal file
View File

@@ -0,0 +1,695 @@
# 知识管理平台自动化工具 - Docker部署版
这是一个基于 Docker 的知识管理平台自动化工具支持多用户、定时任务、代理IP、VIP管理等功能。
---
## 项目简介
本项目是一个 **Docker 容器化应用**,使用 Flask + Playwright + SQLite 构建,提供:
- 多用户注册登录系统
- 浏览器自动化任务
- 定时任务调度
- 截图管理
- VIP用户管理
- 代理IP支持
- 后台管理系统
---
## 技术栈
- **后端**: Python 3.8+, Flask
- **数据库**: SQLite
- **自动化**: Playwright (Chromium)
- **容器化**: Docker + Docker Compose
- **前端**: HTML + JavaScript + Socket.IO
---
## 项目结构
```
zsgpt2/
├── app.py # 主应用程序
├── database.py # 数据库模块
├── playwright_automation.py # 浏览器自动化
├── browser_installer.py # 浏览器安装检查
├── app_config.py # 配置管理
├── app_logger.py # 日志系统
├── app_security.py # 安全模块
├── app_state.py # 状态管理
├── app_utils.py # 工具函数
├── db_pool.py # 数据库连接池
├── password_utils.py # 密码工具
├── requirements.txt # Python依赖
├── Dockerfile # Docker镜像构建文件
├── docker-compose.yml # Docker编排文件
├── templates/ # HTML模板
│ ├── index.html # 主页面
│ ├── login.html # 登录页
│ ├── register.html # 注册页
│ ├── admin.html # 后台管理
│ └── ...
└── static/ # 静态资源
└── js/ # JavaScript文件
```
---
## 部署前准备
### 1. 环境要求
- **服务器**: Linux (Ubuntu 20.04+ / CentOS 7+)
- **Docker**: 20.10+
- **Docker Compose**: 1.29+
- **内存**: 4GB+ (推荐8GB)
- **磁盘**: 20GB+
### 2. SSH连接
**注意**: 本文档假设你已经有服务器的SSH访问权限。
你需要准备:
- 服务器IP地址
- SSH用户名和密码或SSH密钥
- SSH端口默认22
**SSH连接示例**:
```bash
ssh root@your-server-ip
# 或使用密钥
ssh -i /path/to/key root@your-server-ip
```
---
## 快速部署
### 步骤1: 上传项目文件
将整个 `zsgpt2` 文件夹上传到服务器的 `/www/wwwroot/` 目录:
```bash
# 在本地执行Windows PowerShell 或 Git Bash
scp -r C:\Users\Administrator\Desktop\zsgpt2 root@your-server-ip:/www/wwwroot/
# 或者使用 FileZilla、WinSCP 等工具上传
```
上传后,服务器上的路径应该是:`/www/wwwroot/zsgpt2/`
### 步骤2: SSH登录服务器
```bash
ssh root@your-server-ip
```
### 步骤3: 进入项目目录
```bash
cd /www/wwwroot/zsgpt2
```
### 步骤4: 创建必要的目录
```bash
mkdir -p data logs 截图 playwright
chmod 777 data logs 截图 playwright
```
### 步骤5: 构建并启动Docker容器
```bash
# 构建镜像
docker build -t knowledge-automation .
# 启动容器
docker-compose up -d
# 查看容器状态
docker ps | grep knowledge-automation
```
### 步骤6: 检查容器日志
```bash
docker logs -f knowledge-automation-multiuser
```
如果看到以下信息,说明启动成功:
```
服务器启动中...
用户访问地址: http://0.0.0.0:5000
后台管理地址: http://0.0.0.0:5000/yuyx
```
---
## 配置Nginx反向代理可选但推荐
如果你想通过域名访问需要配置Nginx反向代理。
### 1. 安装Nginx
```bash
# Ubuntu/Debian
apt update && apt install nginx -y
# CentOS/RHEL
yum install nginx -y
```
### 2. 创建Nginx配置文件
创建文件 `/etc/nginx/conf.d/zsgpt.conf`:
```nginx
server {
listen 80;
server_name your-domain.com; # 替换为你的域名
# 日志
access_log /var/log/nginx/zsgpt_access.log;
error_log /var/log/nginx/zsgpt_error.log;
# 反向代理
location / {
proxy_pass http://127.0.0.1:5001;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
client_max_body_size 50M;
# WebSocket支持
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_read_timeout 86400;
}
}
```
### 3. 重启Nginx
```bash
nginx -t # 测试配置
nginx -s reload # 重新加载配置
```
### 4. 配置SSL推荐
```bash
# 安装certbot
apt install certbot python3-certbot-nginx -y
# 申请证书
certbot --nginx -d your-domain.com
# 自动续期
certbot renew --dry-run
```
---
## 访问系统
### 用户端
- **HTTP**: `http://your-server-ip:5001`
- **域名**: `http://your-domain.com` (配置Nginx后)
- **HTTPS**: `https://your-domain.com` (配置SSL后)
### 后台管理
- **路径**: `/yuyx`
- **默认账号**: `admin`
- **默认密码**: `admin`
**首次登录后请立即修改密码!**
---
## 系统配置
登录后台后,在"系统配置"页面可以设置:
### 1. 并发控制
- **全局最大并发**: 2 (根据服务器配置调整)
- **单用户并发**: 1
### 2. 定时任务
- **启用定时浏览**: 是/否
- **执行时间**: 02:00 (CST时间)
- **浏览类型**: 应读/注册前未读/未读
- **执行日期**: 周一到周日
### 3. 代理配置
- **启用代理**: 是/否
- **API地址**: http://your-proxy-api.com
- **IP有效期**: 3分钟
---
## Docker常用命令
### 容器管理
```bash
# 启动容器
docker start knowledge-automation-multiuser
# 停止容器
docker stop knowledge-automation-multiuser
# 重启容器
docker restart knowledge-automation-multiuser
# 删除容器
docker rm -f knowledge-automation-multiuser
# 查看容器状态
docker ps -a | grep knowledge-automation
```
### 日志查看
```bash
# 查看实时日志
docker logs -f knowledge-automation-multiuser
# 查看最近100行日志
docker logs --tail 100 knowledge-automation-multiuser
# 查看应用日志文件
tail -f /www/wwwroot/zsgpt2/logs/app.log
```
### 进入容器
```bash
# 进入容器Shell
docker exec -it knowledge-automation-multiuser bash
# 在容器内执行命令
docker exec knowledge-automation-multiuser python -c "print('Hello')"
```
### 重新构建
如果修改了代码,需要重新构建:
```bash
cd /www/wwwroot/zsgpt2
# 停止并删除旧容器
docker-compose down
# 重新构建并启动
docker-compose build
docker-compose up -d
```
---
## 数据备份与恢复
### 1. 备份数据
```bash
cd /www/wwwroot
# 备份整个项目
tar -czf zsgpt2_backup_$(date +%Y%m%d).tar.gz zsgpt2/
# 仅备份数据库
cp /www/wwwroot/zsgpt2/data/app_data.db /backup/app_data_$(date +%Y%m%d).db
# 备份截图
tar -czf screenshots_$(date +%Y%m%d).tar.gz /www/wwwroot/zsgpt2/截图/
```
### 2. 恢复数据
```bash
# 停止容器
docker stop knowledge-automation-multiuser
# 恢复整个项目
cd /www/wwwroot
tar -xzf zsgpt2_backup_20251027.tar.gz
# 恢复数据库
cp /backup/app_data_20251027.db /www/wwwroot/zsgpt2/data/app_data.db
# 重启容器
docker start knowledge-automation-multiuser
```
### 3. 定时备份
添加cron任务自动备份
```bash
crontab -e
```
添加以下内容:
```bash
# 每天凌晨3点备份
0 3 * * * tar -czf /backup/zsgpt2_$(date +\%Y\%m\%d).tar.gz /www/wwwroot/zsgpt2/data
```
---
## 常见问题
### 1. 容器启动失败
**问题**: `docker-compose up -d` 失败
**解决方案**:
```bash
# 查看详细错误
docker-compose logs
# 检查端口占用
netstat -tlnp | grep 5001
# 重新构建
docker-compose build --no-cache
docker-compose up -d
```
### 2. 502 Bad Gateway
**问题**: Nginx返回502错误
**解决方案**:
```bash
# 检查容器是否运行
docker ps | grep knowledge-automation
# 检查端口是否监听
netstat -tlnp | grep 5001
# 测试直接访问
curl http://127.0.0.1:5001
# 检查Nginx配置
nginx -t
```
### 3. 数据库锁定
**问题**: "database is locked"
**解决方案**:
```bash
# 重启容器
docker restart knowledge-automation-multiuser
# 如果问题持续,优化数据库
cd /www/wwwroot/zsgpt2
cp data/app_data.db data/app_data.db.backup
sqlite3 data/app_data.db "VACUUM;"
```
### 4. 内存不足
**问题**: 容器OOM (Out of Memory)
**解决方案**:
修改 `docker-compose.yml`
```yaml
services:
knowledge-automation:
mem_limit: 2g
memswap_limit: 2g
```
然后重启:
```bash
docker-compose down
docker-compose up -d
```
### 5. 浏览器下载失败
**问题**: Playwright浏览器下载失败
**解决方案**:
```bash
# 进入容器手动安装
docker exec -it knowledge-automation-multiuser bash
playwright install chromium
# 或使用国内镜像
export PLAYWRIGHT_DOWNLOAD_HOST=https://npmmirror.com/mirrors/playwright/
playwright install chromium
```
---
## 性能优化
### 1. 调整并发参数
根据服务器配置调整:
- **2核4GB**: 全局并发=1, 单用户并发=1
- **4核8GB**: 全局并发=2, 单用户并发=1
- **8核16GB**: 全局并发=4, 单用户并发=2
### 2. 启用代理IP
避免IP被封提高成功率
- 选择稳定的代理服务商
- 设置合适的IP有效期3-5分钟
- 启用自动重试机制
### 3. 定期清理数据
系统会自动清理7天前的数据也可以手动清理
```bash
# 清理7天前的截图
find /www/wwwroot/zsgpt2/截图 -name "*.jpg" -mtime +7 -delete
# 清理旧日志
find /www/wwwroot/zsgpt2/logs -name "*.log" -mtime +30 -delete
# 优化数据库
sqlite3 /www/wwwroot/zsgpt2/data/app_data.db "VACUUM;"
```
---
## 安全建议
### 1. 修改默认密码
首次登录后立即修改:
- 管理员密码
- 用户密码
### 2. 配置防火墙
```bash
# 只开放必要端口
firewall-cmd --permanent --add-port=80/tcp
firewall-cmd --permanent --add-port=443/tcp
firewall-cmd --reload
# 禁止直接访问5001端口仅Nginx可访问
iptables -A INPUT -p tcp --dport 5001 -s 127.0.0.1 -j ACCEPT
iptables -A INPUT -p tcp --dport 5001 -j DROP
```
### 3. 启用HTTPS
强烈建议使用HTTPS加密传输
```bash
certbot --nginx -d your-domain.com
```
### 4. 限制SSH访问
```bash
# 修改SSH端口可选
vi /etc/ssh/sshd_config
# Port 22222
# 禁止root密码登录使用密钥
PermitRootLogin prohibit-password
PasswordAuthentication no
# 重启SSH服务
systemctl restart sshd
```
---
## 监控与维护
### 1. 系统监控
推荐使用以下工具:
- **Docker Stats**: `docker stats knowledge-automation-multiuser`
- **Grafana + Prometheus**: 可视化监控
- **Uptime Kuma**: 服务可用性监控
### 2. 日志分析
```bash
# 统计今日任务数
grep "浏览完成" /www/wwwroot/zsgpt2/logs/app.log | grep $(date +%Y-%m-%d) | wc -l
# 查看错误日志
grep "ERROR" /www/wwwroot/zsgpt2/logs/app.log | tail -20
# 查看最近的登录
grep "登录成功" /www/wwwroot/zsgpt2/logs/app.log | tail -10
```
### 3. 数据库维护
```bash
# 定期优化数据库(每月一次)
docker exec knowledge-automation-multiuser python3 << 'EOF'
import sqlite3
conn = sqlite3.connect('/app/data/app_data.db')
conn.execute('VACUUM')
conn.close()
print("数据库优化完成")
EOF
```
---
## 更新升级
### 1. 更新代码
```bash
# 停止容器
docker-compose down
# 备份数据
cp -r data data.backup
cp -r 截图 截图.backup
# 上传新代码(覆盖旧文件)
# 使用 scp 或 FTP 工具上传
# 重新构建并启动
docker-compose build
docker-compose up -d
```
### 2. 数据库迁移
如果数据库结构有变化,应用会自动迁移。
查看迁移日志:
```bash
docker logs knowledge-automation-multiuser | grep "数据库"
```
---
## 端口说明
| 端口 | 说明 | 映射 |
|------|------|------|
| 5000 | 容器内应用端口 | - |
| 5001 | 主机映射端口 | 容器5000 → 主机5001 |
| 80 | HTTP端口 | Nginx |
| 443 | HTTPS端口 | Nginx |
---
## 环境变量
可以在 `docker-compose.yml` 中设置的环境变量:
| 变量名 | 说明 | 默认值 |
|--------|------|--------|
| TZ | 时区 | Asia/Shanghai |
| PYTHONUNBUFFERED | Python输出缓冲 | 1 |
| PLAYWRIGHT_BROWSERS_PATH | 浏览器路径 | /ms-playwright |
---
## 技术支持
### 项目信息
- **项目名称**: 知识管理平台自动化工具
- **版本**: Docker 多用户版
- **技术栈**: Python + Flask + Playwright + SQLite + Docker
### 常用文档链接
- [Docker 官方文档](https://docs.docker.com/)
- [Flask 官方文档](https://flask.palletsprojects.com/)
- [Playwright 官方文档](https://playwright.dev/python/)
### 故障排查
遇到问题时,请按以下顺序检查:
1. **容器日志**: `docker logs knowledge-automation-multiuser`
2. **应用日志**: `cat /www/wwwroot/zsgpt2/logs/app.log`
3. **Nginx日志**: `cat /var/log/nginx/zsgpt_error.log`
4. **系统资源**: `docker stats`, `htop`, `df -h`
---
## 许可证
本项目仅供学习和研究使用。
---
**文档版本**: v1.0
**更新日期**: 2025-10-29
**适用版本**: Docker多用户版
---
## 快速上手命令清单
```bash
# 1. 上传文件
scp -r zsgpt2 root@your-ip:/www/wwwroot/
# 2. SSH登录
ssh root@your-ip
# 3. 进入目录并创建必要目录
cd /www/wwwroot/zsgpt2
mkdir -p data logs 截图 playwright
chmod 777 data logs 截图 playwright
# 4. 启动容器
docker-compose up -d
# 5. 查看日志
docker logs -f knowledge-automation-multiuser
# 6. 访问系统
# 浏览器打开: http://your-ip:5001
# 后台管理: http://your-ip:5001/yuyx
# 默认账号: admin / admin
```
完成!🎉

2223
app.py Executable file

File diff suppressed because it is too large Load Diff

182
app_config.py Executable file
View File

@@ -0,0 +1,182 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
配置管理模块
集中管理所有配置项,支持环境变量
"""
import os
from datetime import timedelta
# 常量定义
SECRET_KEY_FILE = 'data/secret_key.txt'
def get_secret_key():
"""获取SECRET_KEY优先环境变量"""
# 优先从环境变量读取
secret_key = os.environ.get('SECRET_KEY')
if secret_key:
return secret_key
# 从文件读取
if os.path.exists(SECRET_KEY_FILE):
with open(SECRET_KEY_FILE, 'r') as f:
return f.read().strip()
# 生成新的
new_key = os.urandom(24).hex()
os.makedirs('data', exist_ok=True)
with open(SECRET_KEY_FILE, 'w') as f:
f.write(new_key)
print(f"✓ 已生成新的SECRET_KEY并保存到 {SECRET_KEY_FILE}")
return new_key
class Config:
"""应用配置基类"""
# ==================== Flask核心配置 ====================
SECRET_KEY = get_secret_key()
# ==================== 会话安全配置 ====================
SESSION_COOKIE_SECURE = os.environ.get('SESSION_COOKIE_SECURE', 'False').lower() == 'true'
SESSION_COOKIE_HTTPONLY = True # 防止XSS攻击
SESSION_COOKIE_SAMESITE = 'Lax' # 防止CSRF攻击
PERMANENT_SESSION_LIFETIME = timedelta(hours=int(os.environ.get('SESSION_LIFETIME_HOURS', '24')))
# ==================== 数据库配置 ====================
DB_FILE = os.environ.get('DB_FILE', 'data/app_data.db')
DB_POOL_SIZE = int(os.environ.get('DB_POOL_SIZE', '5'))
# ==================== 浏览器配置 ====================
SCREENSHOTS_DIR = os.environ.get('SCREENSHOTS_DIR', '截图')
# ==================== 并发控制配置 ====================
MAX_CONCURRENT_GLOBAL = int(os.environ.get('MAX_CONCURRENT_GLOBAL', '2'))
MAX_CONCURRENT_PER_ACCOUNT = int(os.environ.get('MAX_CONCURRENT_PER_ACCOUNT', '1'))
# ==================== 日志缓存配置 ====================
MAX_LOGS_PER_USER = int(os.environ.get('MAX_LOGS_PER_USER', '100'))
MAX_TOTAL_LOGS = int(os.environ.get('MAX_TOTAL_LOGS', '1000'))
# ==================== 验证码配置 ====================
MAX_CAPTCHA_ATTEMPTS = int(os.environ.get('MAX_CAPTCHA_ATTEMPTS', '5'))
CAPTCHA_EXPIRE_SECONDS = int(os.environ.get('CAPTCHA_EXPIRE_SECONDS', '300'))
# ==================== IP限流配置 ====================
MAX_IP_ATTEMPTS_PER_HOUR = int(os.environ.get('MAX_IP_ATTEMPTS_PER_HOUR', '10'))
IP_LOCK_DURATION = int(os.environ.get('IP_LOCK_DURATION', '3600')) # 秒
# ==================== 超时配置 ====================
PAGE_LOAD_TIMEOUT = int(os.environ.get('PAGE_LOAD_TIMEOUT', '60000')) # 毫秒
DEFAULT_TIMEOUT = int(os.environ.get('DEFAULT_TIMEOUT', '60000')) # 毫秒
# ==================== SocketIO配置 ====================
SOCKETIO_CORS_ALLOWED_ORIGINS = os.environ.get('SOCKETIO_CORS_ALLOWED_ORIGINS', '*')
# ==================== 日志配置 ====================
LOG_LEVEL = os.environ.get('LOG_LEVEL', 'INFO')
LOG_FILE = os.environ.get('LOG_FILE', 'logs/app.log')
LOG_MAX_BYTES = int(os.environ.get('LOG_MAX_BYTES', '10485760')) # 10MB
LOG_BACKUP_COUNT = int(os.environ.get('LOG_BACKUP_COUNT', '5'))
# ==================== 安全配置 ====================
DEBUG = os.environ.get('FLASK_DEBUG', 'False').lower() == 'true'
ALLOWED_SCREENSHOT_EXTENSIONS = {'.png', '.jpg', '.jpeg'}
MAX_SCREENSHOT_SIZE = int(os.environ.get('MAX_SCREENSHOT_SIZE', '10485760')) # 10MB
@classmethod
def validate(cls):
"""验证配置的有效性"""
errors = []
# 验证SECRET_KEY
if not cls.SECRET_KEY or len(cls.SECRET_KEY) < 32:
errors.append("SECRET_KEY长度必须至少32个字符")
# 验证并发配置
if cls.MAX_CONCURRENT_GLOBAL < 1:
errors.append("MAX_CONCURRENT_GLOBAL必须大于0")
if cls.MAX_CONCURRENT_PER_ACCOUNT < 1:
errors.append("MAX_CONCURRENT_PER_ACCOUNT必须大于0")
# 验证数据库配置
if not cls.DB_FILE:
errors.append("DB_FILE不能为空")
if cls.DB_POOL_SIZE < 1:
errors.append("DB_POOL_SIZE必须大于0")
# 验证日志配置
if cls.LOG_LEVEL not in ['DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL']:
errors.append(f"LOG_LEVEL无效: {cls.LOG_LEVEL}")
return errors
@classmethod
def print_config(cls):
"""打印当前配置(隐藏敏感信息)"""
print("=" * 60)
print("应用配置")
print("=" * 60)
print(f"DEBUG模式: {cls.DEBUG}")
print(f"SECRET_KEY: {'*' * 20} (长度: {len(cls.SECRET_KEY)})")
print(f"会话超时: {cls.PERMANENT_SESSION_LIFETIME}")
print(f"Cookie安全: HTTPS={cls.SESSION_COOKIE_SECURE}, HttpOnly={cls.SESSION_COOKIE_HTTPONLY}")
print(f"数据库文件: {cls.DB_FILE}")
print(f"数据库连接池: {cls.DB_POOL_SIZE}")
print(f"并发配置: 全局={cls.MAX_CONCURRENT_GLOBAL}, 单账号={cls.MAX_CONCURRENT_PER_ACCOUNT}")
print(f"日志级别: {cls.LOG_LEVEL}")
print(f"日志文件: {cls.LOG_FILE}")
print(f"截图目录: {cls.SCREENSHOTS_DIR}")
print("=" * 60)
class DevelopmentConfig(Config):
"""开发环境配置"""
DEBUG = True
SESSION_COOKIE_SECURE = False
class ProductionConfig(Config):
"""生产环境配置"""
DEBUG = False
SESSION_COOKIE_SECURE = True # 生产环境必须使用HTTPS
class TestingConfig(Config):
"""测试环境配置"""
DEBUG = True
TESTING = True
DB_FILE = 'data/test_app_data.db'
# 根据环境变量选择配置
config_map = {
'development': DevelopmentConfig,
'production': ProductionConfig,
'testing': TestingConfig,
}
def get_config():
"""获取当前环境的配置"""
env = os.environ.get('FLASK_ENV', 'production')
return config_map.get(env, ProductionConfig)
if __name__ == '__main__':
# 配置验证测试
config = get_config()
errors = config.validate()
if errors:
print("配置验证失败:")
for error in errors:
print(f"{error}")
else:
print("✓ 配置验证通过")
config.print_config()

305
app_logger.py Executable file
View File

@@ -0,0 +1,305 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
日志管理模块
提供标准化的日志记录功能
"""
import logging
import os
from logging.handlers import RotatingFileHandler
from datetime import datetime
import threading
# 全局日志配置
_loggers = {}
_logger_lock = threading.Lock()
class ColoredFormatter(logging.Formatter):
"""带颜色的日志格式化器(用于控制台)"""
# ANSI颜色代码
COLORS = {
'DEBUG': '\033[36m', # 青色
'INFO': '\033[32m', # 绿色
'WARNING': '\033[33m', # 黄色
'ERROR': '\033[31m', # 红色
'CRITICAL': '\033[35m', # 紫色
}
RESET = '\033[0m'
def format(self, record):
"""格式化日志记录"""
# 添加颜色
levelname = record.levelname
if levelname in self.COLORS:
record.levelname = f"{self.COLORS[levelname]}{levelname}{self.RESET}"
# 格式化
result = super().format(record)
# 恢复原始levelname避免影响其他handler
record.levelname = levelname
return result
def setup_logger(name='app', level=None, log_file=None, max_bytes=10*1024*1024, backup_count=5):
"""
设置日志记录器
Args:
name: 日志器名称
level: 日志级别DEBUG, INFO, WARNING, ERROR, CRITICAL
log_file: 日志文件路径
max_bytes: 日志文件最大大小(字节)
backup_count: 保留的备份文件数量
Returns:
logging.Logger: 配置好的日志器
"""
with _logger_lock:
# 如果已经存在,直接返回
if name in _loggers:
return _loggers[name]
# 创建日志器
logger = logging.getLogger(name)
# 设置日志级别
if level is None:
level = os.environ.get('LOG_LEVEL', 'INFO')
logger.setLevel(getattr(logging, level.upper()))
# 清除已有的处理器(避免重复)
logger.handlers.clear()
# 日志格式
detailed_formatter = logging.Formatter(
'[%(asctime)s] [%(name)s] [%(levelname)s] [%(filename)s:%(lineno)d] - %(message)s',
datefmt='%Y-%m-%d %H:%M:%S'
)
simple_formatter = logging.Formatter(
'[%(asctime)s] [%(levelname)s] %(message)s',
datefmt='%Y-%m-%d %H:%M:%S'
)
colored_formatter = ColoredFormatter(
'[%(asctime)s] [%(levelname)s] %(message)s',
datefmt='%Y-%m-%d %H:%M:%S'
)
# 控制台处理器(带颜色)
console_handler = logging.StreamHandler()
console_handler.setLevel(logging.INFO)
console_handler.setFormatter(colored_formatter)
logger.addHandler(console_handler)
# 文件处理器(如果指定了文件路径)
if log_file:
# 确保日志目录存在
log_dir = os.path.dirname(log_file)
if log_dir and not os.path.exists(log_dir):
os.makedirs(log_dir, exist_ok=True)
# 主日志文件(详细格式)
file_handler = RotatingFileHandler(
log_file,
maxBytes=max_bytes,
backupCount=backup_count,
encoding='utf-8'
)
file_handler.setLevel(logging.DEBUG)
file_handler.setFormatter(detailed_formatter)
logger.addHandler(file_handler)
# 错误日志文件仅记录WARNING及以上
error_file = log_file.replace('.log', '_error.log')
error_handler = RotatingFileHandler(
error_file,
maxBytes=max_bytes,
backupCount=backup_count,
encoding='utf-8'
)
error_handler.setLevel(logging.WARNING)
error_handler.setFormatter(detailed_formatter)
logger.addHandler(error_handler)
# 防止日志向上传播(避免重复)
logger.propagate = False
# 缓存日志器
_loggers[name] = logger
return logger
def get_logger(name='app'):
"""
获取日志记录器
Args:
name: 日志器名称
Returns:
logging.Logger: 日志器实例
"""
if name in _loggers:
return _loggers[name]
else:
# 如果不存在,创建一个默认的
return setup_logger(name)
class LoggerAdapter:
"""日志适配器,提供便捷的日志记录方法"""
def __init__(self, logger_name='app', context=None):
"""
初始化日志适配器
Args:
logger_name: 日志器名称
context: 上下文信息如用户ID、账号ID等
"""
self.logger = get_logger(logger_name)
self.context = context or {}
def _format_message(self, message):
"""格式化消息,添加上下文信息"""
if self.context:
context_str = ' '.join([f"[{k}={v}]" for k, v in self.context.items()])
return f"{context_str} {message}"
return message
def debug(self, message):
"""记录调试信息"""
self.logger.debug(self._format_message(message))
def info(self, message):
"""记录普通信息"""
self.logger.info(self._format_message(message))
def warning(self, message):
"""记录警告信息"""
self.logger.warning(self._format_message(message))
def error(self, message, exc_info=False):
"""记录错误信息"""
self.logger.error(self._format_message(message), exc_info=exc_info)
def critical(self, message, exc_info=False):
"""记录严重错误信息"""
self.logger.critical(self._format_message(message), exc_info=exc_info)
def exception(self, message):
"""记录异常信息(自动包含堆栈跟踪)"""
self.logger.exception(self._format_message(message))
class AuditLogger:
"""审计日志记录器(用于记录关键操作)"""
def __init__(self, log_file='logs/audit.log'):
"""初始化审计日志"""
self.logger = setup_logger('audit', level='INFO', log_file=log_file)
def log_user_login(self, user_id, username, ip_address, success=True):
"""记录用户登录"""
status = "成功" if success else "失败"
self.logger.info(f"用户登录{status}: user_id={user_id}, username={username}, ip={ip_address}")
def log_admin_login(self, username, ip_address, success=True):
"""记录管理员登录"""
status = "成功" if success else "失败"
self.logger.info(f"管理员登录{status}: username={username}, ip={ip_address}")
def log_user_created(self, user_id, username, created_by=None):
"""记录用户创建"""
self.logger.info(f"用户创建: user_id={user_id}, username={username}, created_by={created_by}")
def log_user_deleted(self, user_id, username, deleted_by):
"""记录用户删除"""
self.logger.warning(f"用户删除: user_id={user_id}, username={username}, deleted_by={deleted_by}")
def log_password_reset(self, user_id, username, reset_by):
"""记录密码重置"""
self.logger.warning(f"密码重置: user_id={user_id}, username={username}, reset_by={reset_by}")
def log_config_change(self, config_name, old_value, new_value, changed_by):
"""记录配置修改"""
self.logger.warning(f"配置修改: {config_name}{old_value} 改为 {new_value}, changed_by={changed_by}")
def log_security_event(self, event_type, description, ip_address=None):
"""记录安全事件"""
self.logger.warning(f"安全事件 [{event_type}]: {description}, ip={ip_address}")
# 全局审计日志实例
audit_logger = AuditLogger()
# 辅助函数
def log_exception(logger, message="发生异常"):
"""记录异常(包含堆栈跟踪)"""
if isinstance(logger, str):
logger = get_logger(logger)
logger.exception(message)
def log_performance(logger, operation, duration_ms, threshold_ms=1000):
"""记录性能信息"""
if isinstance(logger, str):
logger = get_logger(logger)
if duration_ms > threshold_ms:
logger.warning(f"性能警告: {operation} 耗时 {duration_ms}ms (阈值: {threshold_ms}ms)")
else:
logger.debug(f"性能: {operation} 耗时 {duration_ms}ms")
# 初始化默认日志器
def init_logging(log_level='INFO', log_file='logs/app.log'):
"""
初始化日志系统
Args:
log_level: 日志级别
log_file: 日志文件路径
"""
# 创建主应用日志器
setup_logger('app', level=log_level, log_file=log_file)
# 创建数据库日志器
setup_logger('database', level=log_level, log_file='logs/database.log')
# 创建自动化日志器
setup_logger('automation', level=log_level, log_file='logs/automation.log')
# 创建审计日志器已在AuditLogger中创建
print("✓ 日志系统初始化完成")
if __name__ == '__main__':
# 测试日志系统
init_logging(log_level='DEBUG')
logger = get_logger('app')
logger.debug("这是调试信息")
logger.info("这是普通信息")
logger.warning("这是警告信息")
logger.error("这是错误信息")
logger.critical("这是严重错误信息")
# 测试上下文日志
adapter = LoggerAdapter('app', {'user_id': 123, 'username': 'test'})
adapter.info("用户操作日志")
# 测试审计日志
audit_logger.log_user_login(123, 'test_user', '127.0.0.1', success=True)
audit_logger.log_security_event('LOGIN_ATTEMPT', '多次登录失败', '192.168.1.1')
print("\n日志测试完成,请检查 logs/ 目录")

435
app_security.py Executable file
View File

@@ -0,0 +1,435 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
安全工具模块
提供各种安全相关的功能
"""
import os
import re
import time
import hashlib
import secrets
from pathlib import Path
from typing import Optional
from functools import wraps
from flask import request, jsonify, session
from collections import defaultdict
import threading
# ==================== 文件路径安全 ====================
def is_safe_path(basedir, path, follow_symlinks=True):
"""
检查路径是否安全(防止路径遍历攻击)
Args:
basedir: 基础目录
path: 要检查的路径
follow_symlinks: 是否跟随符号链接
Returns:
bool: 路径是否安全
"""
# 检查路径中是否包含危险字符
if '..' in path or path.startswith('/') or path.startswith('\\'):
return False
# 解析路径
if follow_symlinks:
matchpath = os.path.realpath(os.path.join(basedir, path))
else:
matchpath = os.path.abspath(os.path.join(basedir, path))
# 检查是否在基础目录内
return matchpath.startswith(os.path.abspath(basedir))
def sanitize_filename(filename):
"""
清理文件名,移除危险字符
Args:
filename: 原始文件名
Returns:
str: 清理后的文件名
"""
# 移除路径分隔符
filename = filename.replace('/', '_').replace('\\', '_')
# 只保留安全字符
filename = re.sub(r'[^a-zA-Z0-9._-]', '_', filename)
# 限制长度
if len(filename) > 255:
name, ext = os.path.splitext(filename)
filename = name[:255-len(ext)] + ext
return filename
# ==================== IP限流和黑名单 ====================
class IPRateLimiter:
"""IP访问频率限制器"""
def __init__(self, max_attempts=10, window_seconds=3600, lock_duration=3600):
"""
初始化限流器
Args:
max_attempts: 时间窗口内的最大尝试次数
window_seconds: 时间窗口大小(秒)
lock_duration: 锁定时长(秒)
"""
self.max_attempts = max_attempts
self.window_seconds = window_seconds
self.lock_duration = lock_duration
# IP访问记录: {ip: [(timestamp, success), ...]}
self._attempts = defaultdict(list)
# IP锁定记录: {ip: lock_until_timestamp}
self._locked = {}
self._lock = threading.Lock()
def is_locked(self, ip_address):
"""
检查IP是否被锁定
Args:
ip_address: IP地址
Returns:
bool: 是否被锁定
"""
with self._lock:
if ip_address in self._locked:
if time.time() < self._locked[ip_address]:
return True
else:
# 锁定已过期,移除
del self._locked[ip_address]
return False
def record_attempt(self, ip_address, success=True):
"""
记录访问尝试
Args:
ip_address: IP地址
success: 是否成功
Returns:
bool: 是否应该锁定该IP
"""
with self._lock:
now = time.time()
# 清理过期记录
cutoff_time = now - self.window_seconds
self._attempts[ip_address] = [
(ts, succ) for ts, succ in self._attempts[ip_address]
if ts > cutoff_time
]
# 记录本次尝试
self._attempts[ip_address].append((now, success))
# 检查失败次数
failed_attempts = sum(1 for ts, succ in self._attempts[ip_address] if not succ)
if failed_attempts >= self.max_attempts:
# 锁定IP
self._locked[ip_address] = now + self.lock_duration
return True
return False
def get_remaining_attempts(self, ip_address):
"""
获取剩余尝试次数
Args:
ip_address: IP地址
Returns:
int: 剩余尝试次数
"""
with self._lock:
now = time.time()
cutoff_time = now - self.window_seconds
# 清理过期记录
self._attempts[ip_address] = [
(ts, succ) for ts, succ in self._attempts[ip_address]
if ts > cutoff_time
]
failed_attempts = sum(1 for ts, succ in self._attempts[ip_address] if not succ)
return max(0, self.max_attempts - failed_attempts)
def cleanup(self):
"""清理过期数据"""
with self._lock:
now = time.time()
# 清理过期的尝试记录
cutoff_time = now - self.window_seconds
for ip in list(self._attempts.keys()):
self._attempts[ip] = [
(ts, succ) for ts, succ in self._attempts[ip]
if ts > cutoff_time
]
if not self._attempts[ip]:
del self._attempts[ip]
# 清理过期的锁定
for ip in list(self._locked.keys()):
if now >= self._locked[ip]:
del self._locked[ip]
# 全局IP限流器实例
ip_rate_limiter = IPRateLimiter()
def require_ip_not_locked(f):
"""装饰器检查IP是否被锁定"""
@wraps(f)
def decorated_function(*args, **kwargs):
ip_address = request.remote_addr
if ip_rate_limiter.is_locked(ip_address):
return jsonify({
"error": "由于多次失败尝试您的IP已被临时锁定",
"locked_until": ip_rate_limiter._locked.get(ip_address, 0)
}), 429
return f(*args, **kwargs)
return decorated_function
# ==================== 输入验证 ====================
def validate_username(username):
"""
验证用户名格式
Args:
username: 用户名
Returns:
tuple: (is_valid, error_message)
"""
if not username:
return False, "用户名不能为空"
if len(username) < 3:
return False, "用户名长度不能少于3个字符"
if len(username) > 50:
return False, "用户名长度不能超过50个字符"
# 只允许字母、数字、下划线、中文
if not re.match(r'^[\w\u4e00-\u9fa5]+$', username):
return False, "用户名只能包含字母、数字、下划线和中文字符"
return True, None
def validate_password(password):
"""
验证密码强度
Args:
password: 密码
Returns:
tuple: (is_valid, error_message)
"""
if not password:
return False, "密码不能为空"
if len(password) < 6:
return False, "密码长度不能少于6个字符"
if len(password) > 128:
return False, "密码长度不能超过128个字符"
# 可选:强制密码复杂度
# has_upper = bool(re.search(r'[A-Z]', password))
# has_lower = bool(re.search(r'[a-z]', password))
# has_digit = bool(re.search(r'\d', password))
#
# if not (has_upper and has_lower and has_digit):
# return False, "密码必须包含大写字母、小写字母和数字"
return True, None
def validate_email(email):
"""
验证邮箱格式
Args:
email: 邮箱地址
Returns:
tuple: (is_valid, error_message)
"""
if not email:
return True, None # 邮箱可选
# 简单的邮箱正则
pattern = r'^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$'
if not re.match(pattern, email):
return False, "邮箱格式不正确"
if len(email) > 255:
return False, "邮箱长度不能超过255个字符"
return True, None
# ==================== 会话安全 ====================
def generate_session_token():
"""生成安全的会话令牌"""
return secrets.token_urlsafe(32)
def hash_token(token):
"""哈希令牌(用于存储)"""
return hashlib.sha256(token.encode()).hexdigest()
# ==================== CSRF保护 ====================
def generate_csrf_token():
"""生成CSRF令牌"""
if 'csrf_token' not in session:
session['csrf_token'] = secrets.token_urlsafe(32)
return session['csrf_token']
def validate_csrf_token(token):
"""验证CSRF令牌"""
return token == session.get('csrf_token')
# ==================== 内容安全 ====================
def escape_html(text):
"""转义HTML特殊字符防止XSS"""
if not text:
return text
replacements = {
'&': '&amp;',
'<': '&lt;',
'>': '&gt;',
'"': '&quot;',
"'": '&#x27;',
'/': '&#x2F;',
}
for char, escaped in replacements.items():
text = text.replace(char, escaped)
return text
def sanitize_sql_like_pattern(pattern):
"""
清理SQL LIKE模式中的特殊字符
Args:
pattern: LIKE模式字符串
Returns:
str: 清理后的模式
"""
# 转义LIKE中的特殊字符
pattern = pattern.replace('\\', '\\\\')
pattern = pattern.replace('%', '\\%')
pattern = pattern.replace('_', '\\_')
return pattern
# ==================== 安全配置检查 ====================
def check_security_config():
"""
检查安全配置
Returns:
list: 安全问题列表
"""
issues = []
# 检查SECRET_KEY
from flask import current_app
secret_key = current_app.config.get('SECRET_KEY')
if not secret_key or len(secret_key) < 32:
issues.append("SECRET_KEY过短或未设置")
# 检查DEBUG模式
if current_app.config.get('DEBUG'):
issues.append("DEBUG模式在生产环境应该关闭")
# 检查Cookie安全设置
if not current_app.config.get('SESSION_COOKIE_HTTPONLY'):
issues.append("SESSION_COOKIE_HTTPONLY应该设置为True")
if not current_app.config.get('SESSION_COOKIE_SECURE'):
issues.append("生产环境应该启用SESSION_COOKIE_SECURE需要HTTPS")
return issues
# ==================== 辅助函数 ====================
def get_client_ip():
"""
获取客户端真实IP地址
Returns:
str: IP地址
"""
# 检查代理头
if request.headers.get('X-Forwarded-For'):
return request.headers.get('X-Forwarded-For').split(',')[0].strip()
elif request.headers.get('X-Real-IP'):
return request.headers.get('X-Real-IP')
else:
return request.remote_addr
if __name__ == '__main__':
# 测试文件路径安全
print("文件路径安全测试:")
print(f" 安全路径: {is_safe_path('/tmp', 'test.txt')}")
print(f" 危险路径: {is_safe_path('/tmp', '../etc/passwd')}")
# 测试文件名清理
print(f"\n文件名清理: {sanitize_filename('../../../etc/passwd')}")
# 测试输入验证
print("\n输入验证测试:")
print(f" 用户名: {validate_username('test_user')}")
print(f" 密码: {validate_password('Test123456')}")
print(f" 邮箱: {validate_email('test@example.com')}")
# 测试IP限流
print("\nIP限流测试:")
limiter = IPRateLimiter(max_attempts=3, window_seconds=60)
ip = '192.168.1.1'
for i in range(5):
locked = limiter.record_attempt(ip, success=False)
print(f" 尝试 {i+1}: 剩余次数={limiter.get_remaining_attempts(ip)}, 是否锁定={locked}")
print(f" IP被锁定: {limiter.is_locked(ip)}")

328
app_state.py Executable file
View File

@@ -0,0 +1,328 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
应用状态管理模块
提供线程安全的全局状态管理
"""
import threading
from typing import Tuple
from typing import Dict, Any, Optional
from datetime import datetime, timedelta
from app_logger import get_logger
logger = get_logger('app_state')
class ThreadSafeDict:
"""线程安全的字典包装类"""
def __init__(self):
self._dict = {}
self._lock = threading.RLock()
def get(self, key, default=None):
"""获取值"""
with self._lock:
return self._dict.get(key, default)
def set(self, key, value):
"""设置值"""
with self._lock:
self._dict[key] = value
def delete(self, key):
"""删除键"""
with self._lock:
if key in self._dict:
del self._dict[key]
def pop(self, key, default=None):
"""弹出键值"""
with self._lock:
return self._dict.pop(key, default)
def keys(self):
"""获取所有键(返回副本)"""
with self._lock:
return list(self._dict.keys())
def items(self):
"""获取所有键值对(返回副本)"""
with self._lock:
return list(self._dict.items())
def __contains__(self, key):
"""检查键是否存在"""
with self._lock:
return key in self._dict
def clear(self):
"""清空字典"""
with self._lock:
self._dict.clear()
def __len__(self):
"""获取长度"""
with self._lock:
return len(self._dict)
class LogCacheManager:
"""日志缓存管理器(线程安全)"""
def __init__(self, max_logs_per_user=100, max_total_logs=1000):
self._cache = {} # {user_id: [logs]}
self._total_count = 0
self._lock = threading.RLock()
self._max_logs_per_user = max_logs_per_user
self._max_total_logs = max_total_logs
def add_log(self, user_id: int, log_entry: Dict[str, Any]) -> bool:
"""添加日志到缓存"""
with self._lock:
# 检查总数限制
if self._total_count >= self._max_total_logs:
logger.warning(f"日志缓存已满 ({self._max_total_logs}),拒绝添加")
return False
# 初始化用户日志列表
if user_id not in self._cache:
self._cache[user_id] = []
user_logs = self._cache[user_id]
# 检查用户日志数限制
if len(user_logs) >= self._max_logs_per_user:
# 移除最旧的日志
user_logs.pop(0)
self._total_count -= 1
# 添加新日志
user_logs.append(log_entry)
self._total_count += 1
return True
def get_logs(self, user_id: int) -> list:
"""获取用户的所有日志(返回副本)"""
with self._lock:
return list(self._cache.get(user_id, []))
def clear_user_logs(self, user_id: int):
"""清空用户的日志"""
with self._lock:
if user_id in self._cache:
count = len(self._cache[user_id])
del self._cache[user_id]
self._total_count -= count
logger.info(f"清空用户 {user_id}{count} 条日志")
def get_total_count(self) -> int:
"""获取总日志数"""
with self._lock:
return self._total_count
def get_stats(self) -> Dict[str, int]:
"""获取统计信息"""
with self._lock:
return {
'total_count': self._total_count,
'user_count': len(self._cache),
'max_per_user': self._max_logs_per_user,
'max_total': self._max_total_logs
}
class CaptchaManager:
"""验证码管理器(线程安全)"""
def __init__(self, expire_seconds=300):
self._storage = {} # {identifier: {'code': str, 'expire': datetime}}
self._lock = threading.RLock()
self._expire_seconds = expire_seconds
def create(self, identifier: str, code: str) -> None:
"""创建验证码"""
with self._lock:
self._storage[identifier] = {
'code': code,
'expire': datetime.now() + timedelta(seconds=self._expire_seconds)
}
def verify(self, identifier: str, code: str) -> Tuple[bool, str]:
"""验证验证码"""
with self._lock:
if identifier not in self._storage:
return False, "验证码不存在或已过期"
captcha_data = self._storage[identifier]
# 检查是否过期
if datetime.now() > captcha_data['expire']:
del self._storage[identifier]
return False, "验证码已过期,请重新获取"
# 验证码码值
if captcha_data['code'] != code:
return False, "验证码错误"
# 验证成功,删除验证码
del self._storage[identifier]
return True, "验证成功"
def cleanup_expired(self) -> int:
"""清理过期的验证码"""
with self._lock:
now = datetime.now()
expired_keys = [
key for key, data in self._storage.items()
if now > data['expire']
]
for key in expired_keys:
del self._storage[key]
if expired_keys:
logger.info(f"清理了 {len(expired_keys)} 个过期验证码")
return len(expired_keys)
def get_count(self) -> int:
"""获取当前验证码数量"""
with self._lock:
return len(self._storage)
class ApplicationState:
"""应用全局状态管理器(单例模式)"""
_instance = None
_lock = threading.Lock()
def __new__(cls):
if cls._instance is None:
with cls._lock:
if cls._instance is None:
cls._instance = super().__new__(cls)
cls._instance._initialized = False
return cls._instance
def __init__(self):
if self._initialized:
return
# 浏览器管理器
self.browser_manager = None
self._browser_lock = threading.Lock()
# 用户账号管理 {user_id: {account_id: Account对象}}
self.user_accounts = ThreadSafeDict()
# 活动任务管理 {account_id: Thread对象}
self.active_tasks = ThreadSafeDict()
# 日志缓存管理
self.log_cache = LogCacheManager()
# 验证码管理
self.captcha = CaptchaManager()
# 用户信号量管理 {account_id: Semaphore}
self.user_semaphores = ThreadSafeDict()
# 全局信号量
self.global_semaphore = None
self.screenshot_semaphore = threading.Semaphore(1)
self._initialized = True
logger.info("应用状态管理器初始化完成")
def set_browser_manager(self, manager):
"""设置浏览器管理器"""
with self._browser_lock:
self.browser_manager = manager
def get_browser_manager(self):
"""获取浏览器管理器"""
with self._browser_lock:
return self.browser_manager
def get_user_semaphore(self, account_id: int, max_concurrent: int = 1):
"""获取或创建用户信号量"""
if account_id not in self.user_semaphores:
self.user_semaphores.set(account_id, threading.Semaphore(max_concurrent))
return self.user_semaphores.get(account_id)
def set_global_semaphore(self, max_concurrent: int):
"""设置全局信号量"""
self.global_semaphore = threading.Semaphore(max_concurrent)
def get_stats(self) -> Dict[str, Any]:
"""获取状态统计信息"""
return {
'user_accounts_count': len(self.user_accounts),
'active_tasks_count': len(self.active_tasks),
'log_cache_stats': self.log_cache.get_stats(),
'captcha_count': self.captcha.get_count(),
'user_semaphores_count': len(self.user_semaphores),
'browser_manager': 'initialized' if self.browser_manager else 'not_initialized'
}
# 全局单例实例
app_state = ApplicationState()
# 向后兼容的辅助函数
def verify_captcha(identifier: str, code: str) -> Tuple[bool, str]:
"""验证验证码(向后兼容接口)"""
return app_state.captcha.verify(identifier, code)
def create_captcha(identifier: str, code: str) -> None:
"""创建验证码(向后兼容接口)"""
app_state.captcha.create(identifier, code)
def cleanup_expired_captchas() -> int:
"""清理过期验证码(向后兼容接口)"""
return app_state.captcha.cleanup_expired()
if __name__ == '__main__':
# 测试代码
print("测试线程安全状态管理器...")
print("=" * 60)
# 测试 ThreadSafeDict
print("\n1. 测试 ThreadSafeDict:")
td = ThreadSafeDict()
td.set('key1', 'value1')
print(f" 设置 key1 = {td.get('key1')}")
print(f" 长度: {len(td)}")
# 测试 LogCacheManager
print("\n2. 测试 LogCacheManager:")
lcm = LogCacheManager(max_logs_per_user=3, max_total_logs=10)
for i in range(5):
lcm.add_log(1, {'message': f'log {i}'})
print(f" 用户1日志数: {len(lcm.get_logs(1))}")
print(f" 总日志数: {lcm.get_total_count()}")
print(f" 统计: {lcm.get_stats()}")
# 测试 CaptchaManager
print("\n3. 测试 CaptchaManager:")
cm = CaptchaManager(expire_seconds=2)
cm.create('test@example.com', '1234')
success, msg = cm.verify('test@example.com', '1234')
print(f" 验证结果: {success}, {msg}")
# 测试 ApplicationState
print("\n4. 测试 ApplicationState (单例):")
state1 = ApplicationState()
state2 = ApplicationState()
print(f" 单例验证: {state1 is state2}")
print(f" 状态统计: {state1.get_stats()}")
print("\n" + "=" * 60)
print("✓ 所有测试通过!")

302
app_utils.py Executable file
View File

@@ -0,0 +1,302 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
应用工具模块
提取重复的业务逻辑
"""
from typing import Dict, Any, Optional, Tuple
from flask import session, jsonify
from app_logger import get_logger, audit_logger
from app_security import get_client_ip
import database
logger = get_logger('app_utils')
class ValidationError(Exception):
"""验证错误异常"""
pass
def verify_user_file_permission(user_id: int, filename: str) -> Tuple[bool, Optional[str]]:
"""
验证用户文件访问权限
Args:
user_id: 用户ID
filename: 文件名
Returns:
(是否有权限, 错误消息)
"""
# 获取用户信息
user = database.get_user_by_id(user_id)
if not user:
return False, "用户不存在"
username = user['username']
# 检查文件名是否以用户名开头
if not filename.startswith(f"{username}_"):
logger.warning(f"用户 {username} (ID:{user_id}) 尝试访问未授权文件: {filename}")
return False, "无权访问此文件"
return True, None
def log_task_event(account_id: int, status: str, message: str,
browse_type: Optional[str] = None,
screenshot_path: Optional[str] = None) -> bool:
"""
记录任务日志(统一接口)
Args:
account_id: 账号ID
status: 状态running/completed/failed/stopped
message: 消息
browse_type: 浏览类型
screenshot_path: 截图路径
Returns:
是否成功
"""
try:
return database.create_task_log(
account_id=account_id,
status=status,
message=message,
browse_type=browse_type,
screenshot_path=screenshot_path
)
except Exception as e:
logger.error(f"记录任务日志失败: {e}", exc_info=True)
return False
def update_account_status(account_id: int, status: str,
error_message: Optional[str] = None) -> bool:
"""
更新账号状态(统一接口)
Args:
account_id: 账号ID
status: 状态idle/running/error/stopped
error_message: 错误消息仅当status=error时
Returns:
是否成功
"""
try:
return database.update_account_status(
account_id=account_id,
status=status,
error_message=error_message
)
except Exception as e:
logger.error(f"更新账号状态失败 (account_id={account_id}): {e}", exc_info=True)
return False
def get_or_create_config_cache() -> Optional[Dict[str, Any]]:
"""
获取或创建系统配置缓存
缓存存储在session中避免重复查询数据库
Returns:
配置字典失败返回None
"""
# 尝试从session获取缓存
if '_system_config' in session:
return session['_system_config']
# 从数据库加载
try:
config = database.get_system_config()
if config:
# 存入session缓存
session['_system_config'] = config
return config
return None
except Exception as e:
logger.error(f"获取系统配置失败: {e}", exc_info=True)
return None
def clear_config_cache():
"""清除配置缓存(配置变更时调用)"""
if '_system_config' in session:
del session['_system_config']
logger.debug("已清除系统配置缓存")
def safe_close_browser(automation_obj, account_id: int):
"""
安全关闭浏览器(统一错误处理)
Args:
automation_obj: PlaywrightAutomation对象
account_id: 账号ID
"""
if automation_obj:
try:
automation_obj.close()
logger.info(f"账号 {account_id} 的浏览器已关闭")
except Exception as e:
logger.error(f"关闭账号 {account_id} 的浏览器失败: {e}", exc_info=True)
def format_error_response(error: str, status_code: int = 400,
need_captcha: bool = False,
extra_data: Optional[Dict] = None) -> Tuple[Any, int]:
"""
格式化错误响应(统一接口)
Args:
error: 错误消息
status_code: HTTP状态码
need_captcha: 是否需要验证码
extra_data: 额外数据
Returns:
(jsonify响应, 状态码)
"""
response_data = {"error": error}
if need_captcha:
response_data["need_captcha"] = True
if extra_data:
response_data.update(extra_data)
return jsonify(response_data), status_code
def format_success_response(message: str = "操作成功",
extra_data: Optional[Dict] = None) -> Any:
"""
格式化成功响应(统一接口)
Args:
message: 成功消息
extra_data: 额外数据
Returns:
jsonify响应
"""
response_data = {"success": True, "message": message}
if extra_data:
response_data.update(extra_data)
return jsonify(response_data)
def log_user_action(action: str, user_id: int, username: str,
success: bool, details: Optional[str] = None):
"""
记录用户操作到审计日志(统一接口)
Args:
action: 操作类型login/register/logout等
user_id: 用户ID
username: 用户名
success: 是否成功
details: 详细信息
"""
ip = get_client_ip()
if action == 'login':
audit_logger.log_user_login(user_id, username, ip, success)
elif action == 'logout':
audit_logger.log_user_logout(user_id, username, ip)
elif action == 'register':
audit_logger.log_user_created(user_id, username, created_by='self')
if details:
logger.info(f"用户操作: {action}, 用户={username}, 成功={success}, 详情={details}")
def validate_pagination(page: Any, page_size: Any,
max_page_size: int = 100) -> Tuple[int, int, Optional[str]]:
"""
验证分页参数
Args:
page: 页码
page_size: 每页大小
max_page_size: 最大每页大小
Returns:
(页码, 每页大小, 错误消息)
"""
try:
page = int(page) if page else 1
page_size = int(page_size) if page_size else 20
except (ValueError, TypeError):
return 1, 20, "无效的分页参数"
if page < 1:
return 1, 20, "页码必须大于0"
if page_size < 1 or page_size > max_page_size:
return page, 20, f"每页大小必须在1-{max_page_size}之间"
return page, page_size, None
def check_user_ownership(user_id: int, resource_type: str,
resource_id: int) -> Tuple[bool, Optional[str]]:
"""
检查用户是否拥有资源
Args:
user_id: 用户ID
resource_type: 资源类型account/task等
resource_id: 资源ID
Returns:
(是否拥有, 错误消息)
"""
try:
if resource_type == 'account':
account = database.get_account_by_id(resource_id)
if not account:
return False, "账号不存在"
if account['user_id'] != user_id:
return False, "无权访问此账号"
return True, None
elif resource_type == 'task':
# 通过account查询所属用户
# 这里需要根据实际数据库结构实现
pass
return False, "不支持的资源类型"
except Exception as e:
logger.error(f"检查资源所有权失败: {e}", exc_info=True)
return False, "系统错误"
if __name__ == '__main__':
# 测试代码
print("测试应用工具模块...")
print("=" * 60)
# 测试分页验证
print("\n1. 测试分页验证:")
page, page_size, error = validate_pagination("2", "50")
print(f" 页码={page}, 每页={page_size}, 错误={error}")
page, page_size, error = validate_pagination("invalid", "50")
print(f" 无效输入: 页码={page}, 每页={page_size}, 错误={error}")
# 测试响应格式化
print("\n2. 测试响应格式化:")
print(f" 错误响应: {format_error_response('测试错误', need_captcha=True)}")
print(f" 成功响应: {format_success_response('测试成功', {'data': [1, 2, 3]})}")
print("\n" + "=" * 60)
print("✓ 工具模块加载成功!")

214
browser_installer.py Executable file
View File

@@ -0,0 +1,214 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
浏览器自动下载安装模块
检测本地是否有Playwright浏览器如果没有则自动下载安装
"""
import os
import sys
import shutil
import subprocess
from pathlib import Path
# 设置浏览器安装路径支持Docker和本地环境
# Docker环境: PLAYWRIGHT_BROWSERS_PATH环境变量已设置为 /ms-playwright
# 本地环境: 使用Playwright默认路径
if 'PLAYWRIGHT_BROWSERS_PATH' in os.environ:
BROWSERS_PATH = os.environ['PLAYWRIGHT_BROWSERS_PATH']
else:
# Windows: %USERPROFILE%\AppData\Local\ms-playwright
# Linux: ~/.cache/ms-playwright
if sys.platform == 'win32':
BROWSERS_PATH = str(Path.home() / "AppData" / "Local" / "ms-playwright")
else:
BROWSERS_PATH = str(Path.home() / ".cache" / "ms-playwright")
os.environ["PLAYWRIGHT_BROWSERS_PATH"] = BROWSERS_PATH
class BrowserInstaller:
"""浏览器安装器"""
def __init__(self, log_callback=None):
"""
初始化安装器
Args:
log_callback: 日志回调函数
"""
self.log_callback = log_callback
def log(self, message):
"""输出日志"""
if self.log_callback:
self.log_callback(message)
else:
try:
print(message)
except UnicodeEncodeError:
# 如果打印Unicode字符失败替换特殊字符
safe_message = message.replace('', '[OK]').replace('', '[X]')
print(safe_message)
def check_playwright_installed(self):
"""检查Playwright是否已安装"""
try:
import playwright
self.log("✓ Playwright已安装")
return True
except ImportError:
self.log("✗ Playwright未安装")
return False
def check_chromium_installed(self):
"""检查Chromium浏览器是否已安装"""
try:
from playwright.sync_api import sync_playwright
# 尝试启动浏览器检查是否可用
with sync_playwright() as p:
try:
# 使用超时快速检查
browser = p.chromium.launch(headless=True, timeout=5000)
browser.close()
self.log("✓ Chromium浏览器已安装且可用")
return True
except Exception as e:
error_msg = str(e)
self.log(f"✗ Chromium浏览器不可用: {error_msg}")
# 检查是否是路径不存在的错误
if "Executable doesn't exist" in error_msg:
self.log("检测到浏览器文件缺失,需要重新安装")
return False
except Exception as e:
self.log(f"✗ 检查浏览器时出错: {str(e)}")
return False
def install_chromium(self):
"""安装Chromium浏览器"""
try:
self.log("正在安装 Chromium 浏览器...")
# 查找 playwright 可执行文件
playwright_cli = None
possible_paths = [
os.path.join(os.path.dirname(sys.executable), "Scripts", "playwright.exe"),
os.path.join(os.path.dirname(sys.executable), "playwright.exe"),
os.path.join(os.path.dirname(sys.executable), "Scripts", "playwright"),
os.path.join(os.path.dirname(sys.executable), "playwright"),
"playwright", # 系统PATH中
]
for path in possible_paths:
if os.path.exists(path) or shutil.which(path):
playwright_cli = path
break
# 如果找到了 playwright CLI直接调用
if playwright_cli:
self.log(f"使用 Playwright CLI: {playwright_cli}")
result = subprocess.run(
[playwright_cli, "install", "chromium"],
capture_output=True,
text=True,
timeout=300
)
else:
# 检测是否是 Nuitka 编译的程序
is_nuitka = hasattr(sys, 'frozen') or '__compiled__' in globals()
if is_nuitka:
self.log("检测到 Nuitka 编译环境")
self.log("✗ 无法找到 playwright CLI 工具")
self.log("请手动运行: playwright install chromium")
return False
else:
# 使用 python -m
result = subprocess.run(
[sys.executable, "-m", "playwright", "install", "chromium"],
capture_output=True,
text=True,
timeout=300
)
if result.returncode == 0:
self.log("✓ Chromium浏览器安装成功")
return True
else:
self.log(f"✗ 浏览器安装失败: {result.stderr}")
return False
except subprocess.TimeoutExpired:
self.log("✗ 浏览器安装超时")
return False
except Exception as e:
self.log(f"✗ 浏览器安装出错: {str(e)}")
return False
def auto_install(self):
"""
自动检测并安装所需环境
Returns:
是否成功安装或已安装
"""
self.log("=" * 60)
self.log("检查浏览器环境...")
self.log("=" * 60)
# 1. 检查Playwright是否安装
if not self.check_playwright_installed():
self.log("✗ Playwright未安装无法继续")
self.log("请确保程序包含 Playwright 库")
return False
# 2. 检查Chromium浏览器是否安装
if not self.check_chromium_installed():
self.log("\n未检测到Chromium浏览器开始自动安装...")
# 安装浏览器
if not self.install_chromium():
self.log("✗ 浏览器安装失败")
self.log("\n您可以尝试以下方法:")
self.log("1. 手动执行: playwright install chromium")
self.log("2. 检查网络连接后重试")
self.log("3. 检查防火墙设置")
return False
self.log("\n" + "=" * 60)
self.log("✓ 浏览器环境检查完成,一切就绪!")
self.log("=" * 60 + "\n")
return True
def check_and_install_browser(log_callback=None):
"""
便捷函数:检查并安装浏览器
Args:
log_callback: 日志回调函数
Returns:
是否成功
"""
installer = BrowserInstaller(log_callback)
return installer.auto_install()
# 测试代码
if __name__ == "__main__":
print("浏览器自动安装工具")
print("=" * 60)
installer = BrowserInstaller()
success = installer.auto_install()
if success:
print("\n✓ 安装成功!您现在可以运行主程序了。")
else:
print("\n✗ 安装失败,请查看上方错误信息。")
print("=" * 60)

1066
database.py Executable file

File diff suppressed because it is too large Load Diff

250
db_pool.py Executable file
View File

@@ -0,0 +1,250 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
数据库连接池模块
使用queue实现固定大小的连接池,防止连接泄漏
"""
import sqlite3
import threading
from queue import Queue, Empty
import time
class ConnectionPool:
"""SQLite连接池"""
def __init__(self, database, pool_size=5, timeout=30):
"""
初始化连接池
Args:
database: 数据库文件路径
pool_size: 连接池大小(默认5)
timeout: 获取连接超时时间(秒)
"""
self.database = database
self.pool_size = pool_size
self.timeout = timeout
self._pool = Queue(maxsize=pool_size)
self._lock = threading.Lock()
self._created_connections = 0
# 预创建连接
self._initialize_pool()
def _initialize_pool(self):
"""预创建连接池中的连接"""
for _ in range(self.pool_size):
conn = self._create_connection()
self._pool.put(conn)
self._created_connections += 1
def _create_connection(self):
"""创建新的数据库连接"""
conn = sqlite3.connect(self.database, check_same_thread=False)
conn.row_factory = sqlite3.Row
# 设置WAL模式提高并发性能
conn.execute('PRAGMA journal_mode=WAL')
# 设置合理的超时时间
conn.execute('PRAGMA busy_timeout=5000')
return conn
def get_connection(self):
"""
从连接池获取连接
Returns:
PooledConnection: 连接包装对象
"""
try:
conn = self._pool.get(timeout=self.timeout)
return PooledConnection(conn, self)
except Empty:
raise RuntimeError(f"无法在{self.timeout}秒内获取数据库连接")
def return_connection(self, conn):
"""
归还连接到连接池 [已修复Bug#7]
Args:
conn: 要归还的连接
"""
import sqlite3
from queue import Full
try:
# 回滚任何未提交的事务
conn.rollback()
self._pool.put(conn, block=False)
except sqlite3.Error as e:
# 数据库相关错误,连接可能损坏
print(f"归还连接失败(数据库错误): {e}")
try:
conn.close()
except Exception:
pass
# 创建新连接补充
with self._lock:
try:
new_conn = self._create_connection()
self._pool.put(new_conn, block=False)
except Exception as create_error:
print(f"重建连接失败: {create_error}")
except Full:
# 队列已满(不应该发生)
print(f"警告: 连接池已满,关闭多余连接")
try:
conn.close()
except Exception:
pass
except Exception as e:
print(f"归还连接失败(未知错误): {e}")
try:
conn.close()
except Exception:
pass
def close_all(self):
"""关闭所有连接"""
while not self._pool.empty():
try:
conn = self._pool.get(block=False)
conn.close()
except Exception as e:
print(f"关闭连接失败: {e}")
def get_stats(self):
"""获取连接池统计信息"""
return {
'pool_size': self.pool_size,
'available': self._pool.qsize(),
'in_use': self.pool_size - self._pool.qsize(),
'total_created': self._created_connections
}
class PooledConnection:
"""连接池连接包装器,支持with语句自动归还"""
def __init__(self, conn, pool):
"""
初始化
Args:
conn: 实际的数据库连接
pool: 连接池对象
"""
self._conn = conn
self._pool = pool
self._cursor = None
def __enter__(self):
"""支持with语句"""
return self
def __exit__(self, exc_type, exc_val, exc_tb):
"""with语句结束时自动归还连接 [已修复Bug#3]"""
try:
if exc_type is not None:
# 发生异常,回滚事务
self._conn.rollback()
print(f"数据库事务已回滚: {exc_type.__name__}")
# 注意: 不自动commit要求用户显式调用conn.commit()
if self._cursor:
self._cursor.close()
except Exception as e:
print(f"关闭游标失败: {e}")
finally:
# 归还连接
self._pool.return_connection(self._conn)
return False # 不抑制异常
def cursor(self):
"""获取游标"""
self._cursor = self._conn.cursor()
return self._cursor
def commit(self):
"""提交事务"""
self._conn.commit()
def rollback(self):
"""回滚事务"""
self._conn.rollback()
def execute(self, sql, parameters=None):
"""执行SQL"""
cursor = self.cursor()
if parameters:
return cursor.execute(sql, parameters)
return cursor.execute(sql)
def fetchone(self):
"""获取一行"""
if self._cursor:
return self._cursor.fetchone()
return None
def fetchall(self):
"""获取所有行"""
if self._cursor:
return self._cursor.fetchall()
return []
@property
def lastrowid(self):
"""最后插入的行ID"""
if self._cursor:
return self._cursor.lastrowid
return None
@property
def rowcount(self):
"""影响的行数"""
if self._cursor:
return self._cursor.rowcount
return 0
# 全局连接池实例
_pool = None
_pool_lock = threading.Lock()
def init_pool(database, pool_size=5):
"""
初始化全局连接池
Args:
database: 数据库文件路径
pool_size: 连接池大小
"""
global _pool
with _pool_lock:
if _pool is None:
_pool = ConnectionPool(database, pool_size)
print(f"✓ 数据库连接池已初始化 (大小: {pool_size})")
def get_db():
"""
获取数据库连接(替代原有的get_db函数)
Returns:
PooledConnection: 连接对象
"""
global _pool
if _pool is None:
raise RuntimeError("连接池未初始化,请先调用init_pool()")
return _pool.get_connection()
def get_pool_stats():
"""获取连接池统计信息"""
global _pool
if _pool:
return _pool.get_stats()
return None

20
docker-compose.yml Normal file
View File

@@ -0,0 +1,20 @@
version: '3.8'
services:
knowledge-automation:
build: .
container_name: knowledge-automation-multiuser
ports:
- "5001:5000"
volumes:
- ./data:/app/data # 数据库持久化
- ./logs:/app/logs # 日志持久化
- ./截图:/app/截图 # 截图持久化
- ./playwright:/ms-playwright # Playwright浏览器持久化避免重复下载
- /etc/localtime:/etc/localtime:ro # 时区同步
environment:
- TZ=Asia/Shanghai
- PYTHONUNBUFFERED=1
- PLAYWRIGHT_BROWSERS_PATH=/ms-playwright
restart: unless-stopped
shm_size: 2gb # 为Chromium分配共享内存

0
ftp-manager.db Normal file
View File

74
password_utils.py Normal file
View File

@@ -0,0 +1,74 @@
"""
密码哈希工具模块
支持bcrypt加密和SHA256兼容性验证
"""
import bcrypt
import hashlib
def hash_password_bcrypt(password):
"""
使用bcrypt加密密码
Args:
password: 明文密码
Returns:
str: bcrypt哈希值(包含盐值)
"""
salt = bcrypt.gensalt(rounds=12)
return bcrypt.hashpw(password.encode('utf-8'), salt).decode('utf-8')
def verify_password_bcrypt(password, password_hash):
"""
验证bcrypt密码
Args:
password: 明文密码
password_hash: bcrypt哈希值
Returns:
bool: 验证成功返回True
"""
try:
return bcrypt.checkpw(password.encode('utf-8'),
password_hash.encode('utf-8'))
except Exception as e:
print(f"bcrypt验证异常: {e}")
return False
def is_sha256_hash(password_hash):
"""
判断是否为旧的SHA256哈希
Args:
password_hash: 哈希值
Returns:
bool: SHA256哈希为64位十六进制字符串
"""
if not password_hash:
return False
# SHA256输出固定64位十六进制
return len(password_hash) == 64 and all(c in '0123456789abcdef' for c in password_hash.lower())
def verify_password_sha256(password, password_hash):
"""
验证旧的SHA256密码(兼容性)
Args:
password: 明文密码
password_hash: SHA256哈希值
Returns:
bool: 验证成功返回True
"""
try:
computed_hash = hashlib.sha256(password.encode()).hexdigest()
return computed_hash == password_hash
except Exception as e:
print(f"SHA256验证异常: {e}")
return False

762
playwright_automation.py Executable file
View File

@@ -0,0 +1,762 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Playwright版本 - 知识管理系统自动化核心
使用浏览器上下文(Context)实现高性能并发
"""
import os
from pathlib import Path
from playwright.sync_api import sync_playwright, Browser, BrowserContext, Page, Playwright
import time
import threading
from typing import Optional, Callable
from dataclasses import dataclass
# 设置浏览器安装路径避免Nuitka onefile临时目录问题
BROWSERS_PATH = str(Path.home() / "AppData" / "Local" / "ms-playwright")
os.environ["PLAYWRIGHT_BROWSERS_PATH"] = BROWSERS_PATH
# 配置常量
class Config:
"""配置常量"""
LOGIN_URL = "https://postoa.aidunsoft.com/admin/login.aspx"
INDEX_URL_PATTERN = "index.aspx"
PAGE_LOAD_TIMEOUT = 60000 # 毫秒 (increased from 30s to 60s for multi-account support)
DEFAULT_TIMEOUT = 60000 # 增加超时时间以支持多账号并发
MAX_CONCURRENT_CONTEXTS = 100 # 最大并发上下文数
@dataclass
class BrowseResult:
"""浏览结果"""
success: bool
total_items: int = 0
total_attachments: int = 0
error_message: str = ""
class PlaywrightBrowserManager:
"""Playwright浏览器管理器 - 每个账号独立的浏览器实例"""
def __init__(self, headless: bool = True, log_callback: Optional[Callable] = None):
"""
初始化浏览器管理器
Args:
headless: 是否使用无头模式
log_callback: 日志回调函数,签名: log_callback(message, account_id=None)
"""
self.headless = headless
self.log_callback = log_callback
self._lock = threading.Lock()
def log(self, message: str, account_id: Optional[str] = None):
"""记录日志"""
if self.log_callback:
self.log_callback(message, account_id)
def create_browser(self, proxy_config=None):
"""创建新的独立浏览器实例(每个账号独立)"""
try:
self.log("初始化Playwright实例...")
playwright = sync_playwright().start()
self.log("启动独立浏览器进程...")
start_time = time.time()
# 准备浏览器启动参数
launch_options = {
'headless': self.headless,
'args': [
'--no-sandbox',
'--disable-dev-shm-usage',
'--disable-gpu',
'--disable-extensions',
'--disable-notifications',
'--disable-infobars',
'--disable-default-apps',
'--disable-background-timer-throttling',
'--disable-backgrounding-occluded-windows',
'--disable-renderer-backgrounding',
]
}
# 如果有代理配置,添加代理
if proxy_config and proxy_config.get('server'):
launch_options['proxy'] = {
'server': proxy_config['server']
}
self.log(f"使用代理: {proxy_config['server']}")
browser = playwright.chromium.launch(**launch_options)
elapsed = time.time() - start_time
self.log(f"独立浏览器启动成功 (耗时: {elapsed:.2f}秒)")
return playwright, browser
except Exception as e:
self.log(f"启动浏览器失败: {str(e)}")
raise
def create_browser_and_context(self, proxy_config=None):
"""创建独立的浏览器和上下文(每个账号完全隔离)"""
playwright, browser = self.create_browser(proxy_config)
start_time = time.time()
self.log("创建浏览器上下文...")
context = browser.new_context(
viewport={'width': 1920, 'height': 1080},
user_agent='Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36',
device_scale_factor=2, # 2倍设备像素比提高文字清晰度
)
# 设置默认超时
context.set_default_timeout(Config.DEFAULT_TIMEOUT)
context.set_default_navigation_timeout(Config.PAGE_LOAD_TIMEOUT)
elapsed = time.time() - start_time
self.log(f"上下文创建完成 (耗时: {elapsed:.3f}秒)")
return playwright, browser, context
class PlaywrightAutomation:
"""Playwright自动化操作类"""
def __init__(self, browser_manager: PlaywrightBrowserManager, account_id: str, proxy_config: Optional[dict] = None):
"""
初始化自动化操作
Args:
browser_manager: 浏览器管理器
account_id: 账号ID用于日志
"""
self.browser_manager = browser_manager
self.account_id = account_id
self.proxy_config = proxy_config
self.playwright: Optional[Playwright] = None
self.browser: Optional[Browser] = None
self.context: Optional[BrowserContext] = None
self.page: Optional[Page] = None
self.main_page: Optional[Page] = None
def log(self, message: str):
"""记录日志"""
self.browser_manager.log(message, self.account_id)
def login(self, username: str, password: str, remember: bool = True) -> bool:
"""
登录系统
Args:
username: 用户名
password: 密码
remember: 是否记住密码
Returns:
是否登录成功
"""
try:
self.log("创建浏览器上下文...")
start_time = time.time()
self.playwright, self.browser, self.context = self.browser_manager.create_browser_and_context(self.proxy_config)
elapsed = time.time() - start_time
self.log(f"浏览器和上下文创建完成 (耗时: {elapsed:.3f}秒)")
self.log("创建页面...")
self.page = self.context.new_page()
self.main_page = self.page
self.log("访问登录页面...")
# 使用重试机制处理超时
max_retries = 2
for attempt in range(max_retries):
try:
self.page.goto(Config.LOGIN_URL, timeout=60000)
break
except Exception as e:
if attempt < max_retries - 1:
self.log(f"页面加载超时,重试中... ({attempt + 1}/{max_retries})")
time.sleep(2)
else:
raise
self.log("填写登录信息...")
self.page.fill('#txtUserName', username)
self.page.fill('#txtPassword', password)
if remember:
self.page.check('#chkRemember')
self.log("点击登录按钮...")
self.page.click('#btnSubmit')
# 等待跳转
self.log("等待登录处理...")
self.page.wait_for_load_state('networkidle', timeout=30000) # 增加到30秒
# 检查登录结果
current_url = self.page.url
self.log(f"当前URL: {current_url}")
if Config.INDEX_URL_PATTERN in current_url:
self.log("登录成功!")
return True
else:
self.log("登录失败,请检查用户名和密码")
return False
except Exception as e:
self.log(f"登录过程中出错: {str(e)}")
return False
def switch_to_iframe(self) -> bool:
"""切换到mainframe iframe"""
try:
self.log("查找并切换到iframe...")
# 使用Playwright的等待机制
max_retries = 3
for i in range(max_retries):
try:
# 等待iframe元素出现
self.main_page.wait_for_selector("iframe[name='mainframe']", timeout=2000)
# 获取iframe
iframe = self.main_page.frame('mainframe')
if iframe:
self.page = iframe
self.log(f"✓ 成功切换到iframe (尝试 {i+1}/{max_retries})")
return True
except Exception as e:
if i < max_retries - 1:
self.log(f"未找到iframe重试中... ({i+1}/{max_retries})")
time.sleep(1)
else:
self.log(f"所有重试都失败未找到iframe")
return False
except Exception as e:
self.log(f"切换到iframe时出错: {str(e)}")
return False
def switch_browse_type(self, browse_type: str, max_retries: int = 2) -> bool:
"""
切换浏览类型(带重试机制)
Args:
browse_type: 浏览类型(注册前未读/应读/已读)
max_retries: 最大重试次数(默认2次)
Returns:
是否切换成功
"""
for attempt in range(max_retries + 1):
try:
if attempt > 0:
self.log(f"⚠ 第 {attempt + 1} 次尝试切换浏览类型...")
else:
self.log(f"切换到'{browse_type}'类型...")
# 切换到iframe
if not self.switch_to_iframe():
if attempt < max_retries:
self.log(f"iframe切换失败,等待1秒后重试...")
time.sleep(1)
continue
return False
# 方法1: 尝试查找<a>标签如果JavaScript创建了的话
selector = f"//div[contains(@class, 'rule-multi-radio')]//a[contains(text(), '{browse_type}')]"
try:
# 等待并点击
self.page.locator(selector).click(timeout=5000)
self.log(f"点击'{browse_type}'按钮成功")
# 等待页面刷新并加载内容
time.sleep(1.5)
# 等待表格加载最多等待30秒
try:
self.page.locator("//table[@class='ltable']").wait_for(timeout=30000)
self.log("内容表格已加载")
except Exception as e:
self.log("等待表格加载超时,继续...")
return True
except Exception as e:
error_msg = str(e)
if "Execution context was destroyed" in error_msg:
self.log(f"⚠ 检测到执行上下文被销毁")
if attempt < max_retries:
self.log(f"等待2秒后重试...")
time.sleep(2)
continue
self.log(f"未找到<a>标签,尝试点击<label>...")
# 方法2: 点击label模拟点击radio button
label_selector = f"//label[contains(text(), '{browse_type}')]"
try:
self.page.locator(label_selector).click(timeout=5000)
self.log(f"点击'{browse_type}'标签成功")
# 等待页面刷新并加载内容
time.sleep(1.5)
# 等待表格加载最多等待30秒
try:
self.page.locator("//table[@class='ltable']").wait_for(timeout=30000)
self.log("内容表格已加载")
except Exception as e:
self.log("等待表格加载超时,继续...")
return True
except Exception as e:
error_msg = str(e)
if "Execution context was destroyed" in error_msg:
self.log(f"⚠ 检测到执行上下文被销毁")
if attempt < max_retries:
self.log(f"等待2秒后重试...")
time.sleep(2)
continue
self.log(f"未找到<label>标签")
# 如果两种方法都失败,但还有重试机会
if attempt < max_retries:
self.log(f"切换失败,等待2秒后重试...")
time.sleep(2)
continue
return False
except Exception as e:
error_msg = str(e)
self.log(f"切换浏览类型时出错: {error_msg}")
# 检查是否是 "Execution context was destroyed" 错误
if "Execution context was destroyed" in error_msg or "navigation" in error_msg.lower():
if attempt < max_retries:
self.log(f"⚠ 检测到执行上下文被销毁或导航错误,等待2秒后重试...")
time.sleep(2)
continue
return False
# 所有重试都失败
self.log(f"❌ 切换浏览类型失败,已重试 {max_retries}")
return False
def browse_content(self, browse_type: str,
auto_next_page: bool = True,
auto_view_attachments: bool = True,
interval: float = 1.0,
should_stop_callback: Optional[Callable] = None) -> BrowseResult:
"""
浏览内容
Args:
browse_type: 浏览类型
auto_next_page: 是否自动翻页
auto_view_attachments: 是否自动查看附件
interval: 查看附件的间隔时间(秒)
should_stop_callback: 检查是否应该停止的回调函数
Returns:
浏览结果
"""
result = BrowseResult(success=False)
try:
# 先导航到浏览页面
self.log(f"导航到 '{browse_type}' 页面...")
try:
# 等待页面完全加载
time.sleep(2)
self.log(f"当前URL: {self.main_page.url}")
except Exception as e:
self.log(f"获取URL失败: {str(e)}")
# 切换浏览类型
if not self.switch_browse_type(browse_type):
result.error_message = "切换浏览类型失败"
return result
current_page = 1
total_items = 0
total_attachments = 0
completed_first_round = False
empty_page_counter = 0
while True:
# 检查是否应该停止
if should_stop_callback and should_stop_callback():
self.log("收到停止信号,终止浏览")
break
self.log(f"处理第 {current_page} 页...")
# 确保在iframe中(关键!)
time.sleep(0.2)
self.page = self.main_page.frame('mainframe')
if not self.page:
self.log("错误无法获取iframe")
break
# 额外等待确保AJAX内容加载完成
time.sleep(0.5)
# 获取内容行数量
rows_locator = self.page.locator("//table[@class='ltable']/tbody/tr[position()>1 and count(td)>=5]")
rows_count = rows_locator.count()
if rows_count == 0:
self.log("当前页面没有内容")
empty_page_counter += 1
self.log(f"连续空页面数: {empty_page_counter}")
# 检查是否已完成至少一轮浏览且连续空页面数达到阈值
if completed_first_round and empty_page_counter >= 2:
self.log("检测到连续空页面且已完成至少一轮浏览,内容已浏览完毕")
break
# 尝试翻页或返回第一页
if auto_next_page:
# 检查是否有下一页
try:
next_button = self.page.locator("//div[@id='PageContent']/a[contains(text(), '下一页') or contains(text(), '»')]")
if next_button.count() > 0:
self.log("点击下一页...")
next_button.click()
time.sleep(1.5)
current_page += 1
continue
else:
# 没有下一页,返回第一页
if not completed_first_round:
completed_first_round = True
self.log("完成第一轮浏览,准备返回第一页继续浏览...")
else:
self.log("完成一轮浏览,返回第一页继续...")
# 刷新页面并重新点击浏览类型
self.log("刷新页面并重新点击浏览类型...")
self.main_page.reload()
time.sleep(1.5)
# 切换到iframe
time.sleep(0.5)
self.page = self.main_page.frame('mainframe')
# 重新点击浏览类型按钮
selector = f"//div[contains(@class, 'rule-multi-radio')]//a[contains(text(), '{browse_type}')]"
try:
self.page.locator(selector).click(timeout=5000)
self.log(f"重新点击'{browse_type}'按钮成功")
time.sleep(1.5)
# 等待表格加载
try:
self.page.locator("//table[@class='ltable']").wait_for(timeout=30000) # 增加到30秒
self.log("内容表格已加载")
except Exception as e:
self.log("等待表格加载超时,继续...")
except Exception as e:
# 尝试点击label
label_selector = f"//label[contains(text(), '{browse_type}')]"
self.page.locator(label_selector).click(timeout=5000)
self.log(f"点击'{browse_type}'标签成功")
time.sleep(1.5)
current_page = 1
continue
except Exception as e:
self.log(f"翻页时出错: {str(e)}")
break
else:
break
# 找到内容,重置空页面计数
empty_page_counter = 0
self.log(f"找到 {rows_count} 条内容")
# 处理每一行 (每次从头重新获取所有行)
for i in range(rows_count):
if should_stop_callback and should_stop_callback():
break
# 每次处理新行前,确保在iframe中(关键!尤其是history.back()后)
if i > 0:
time.sleep(0.2)
self.page = self.main_page.frame('mainframe')
# 每次都重新获取rows_locator和row,确保元素是最新的
current_rows_locator = self.page.locator("//table[@class='ltable']/tbody/tr[position()>1 and count(td)>=5]")
row = current_rows_locator.nth(i)
# 获取标题 (使用xpath:)
title_cell = row.locator("xpath=.//td[4]")
title = title_cell.inner_text().strip()
self.log(f" [{i+1}] {title[:50]}")
total_items += 1
# 处理附件 (使用xpath:)
if auto_view_attachments:
# 每次都重新获取附件链接数量
att_links_locator = row.locator("xpath=.//td[5]//a[contains(@class, 'link-btn')]")
att_count = att_links_locator.count()
if att_count > 0:
# 只处理第一个附件
att_link = att_links_locator.first
att_text = att_link.inner_text().strip() or "附件"
self.log(f" - 处理{att_text}...")
try:
# 记录点击前的页面数量
pages_before = len(self.context.pages)
# 点击附件
att_link.click()
# 快速检测是否有新窗口0.5秒足够)
time.sleep(0.5)
# 检查是否有新窗口
pages_after = self.context.pages
if len(pages_after) > pages_before:
# 有新窗口打开
new_page = pages_after[-1]
self.log(f" - 新窗口已打开,等待加载...")
time.sleep(interval) # 使用用户设置的间隔
# 关闭新窗口
new_page.close()
self.log(f" - 新窗口已关闭")
else:
# 没有新窗口使用浏览器返回像Selenium版本一样
# 关键问题iframe内点击附件不会触发真正的导航
# Selenium的driver.back()不等待Playwright的go_back()会等待导航
# 解决方案使用JavaScript执行history.back(),不等待导航
self.main_page.evaluate("() => window.history.back()")
time.sleep(0.5)
# 确保回到iframe中
self.page = self.main_page.frame('mainframe')
# 确保回到iframe中
time.sleep(0.2)
self.page = self.main_page.frame('mainframe')
total_attachments += 1
self.log(f" - {att_text}处理完成")
except Exception as e:
self.log(f" - 处理{att_text}时出错: {str(e)}")
# 发生错误时尝试恢复到iframe
try:
# 尝试重新获取iframe
iframe = self.main_page.frame('mainframe')
if iframe:
self.page = iframe
else:
# 如果找不到iframe可能需要刷新
self.log(f" - 找不到iframe刷新页面...")
self.main_page.reload()
time.sleep(1)
if self.switch_browse_type(browse_type):
self.page = self.main_page.frame('mainframe')
except Exception as e:
pass
# 处理完当前页后,检查是否需要翻页
if auto_next_page:
try:
# 确保在iframe中
time.sleep(0.2)
self.page = self.main_page.frame('mainframe')
# 检查是否有下一页
next_button = self.page.locator("//div[@id='PageContent']/a[contains(text(), '下一页') or contains(text(), '»')]")
if next_button.count() > 0:
self.log("点击下一页...")
next_button.click()
time.sleep(1.5)
current_page += 1
# 继续下一页的循环
else:
# 没有下一页了,返回第一页继续
if not completed_first_round:
completed_first_round = True
self.log("完成第一轮浏览,准备返回第一页继续浏览...")
else:
self.log("完成一轮浏览,返回第一页继续...")
# 刷新页面并重新点击浏览类型
self.log("刷新页面并重新点击浏览类型...")
self.main_page.reload()
time.sleep(1.5)
# 切换到iframe
time.sleep(0.5)
self.page = self.main_page.frame('mainframe')
# 重新点击浏览类型按钮
selector = f"//div[contains(@class, 'rule-multi-radio')]//a[contains(text(), '{browse_type}')]"
try:
self.page.locator(selector).click(timeout=5000)
self.log(f"重新点击'{browse_type}'按钮成功")
time.sleep(1.5)
# 等待表格加载
try:
self.page.locator("//table[@class='ltable']").wait_for(timeout=30000) # 增加到30秒
self.log("内容表格已加载")
except Exception as e:
self.log("等待表格加载超时,继续...")
except Exception as e:
# 尝试点击label
label_selector = f"//label[contains(text(), '{browse_type}')]"
self.page.locator(label_selector).click(timeout=5000)
self.log(f"点击'{browse_type}'标签成功")
time.sleep(1.5)
current_page = 1
# 继续循环,从第一页开始
except Exception as e:
self.log(f"翻页时出错: {str(e)}")
break
result.success = True
result.total_items = total_items
result.total_attachments = total_attachments
self.log(f"浏览完成!共 {total_items} 条内容,{total_attachments} 个附件")
except Exception as e:
result.error_message = str(e)
self.log(f"浏览内容时出错: {str(e)}")
return result
def take_screenshot(self, filepath: str) -> bool:
"""
截图
Args:
filepath: 截图保存路径
Returns:
是否截图成功
"""
try:
# 使用最高质量设置截图
# type='jpeg' 指定JPEG格式支持quality参数
# quality=100 表示100%的JPEG质量范围0-100最高质量
# full_page=True 表示截取整个页面
# 视口分辨率 2560x1440 确保高清晰度
# 这样可以生成更清晰的截图大小约500KB-1MB左右
self.main_page.screenshot(
path=filepath,
type='jpeg',
full_page=True,
quality=100
)
self.log(f"截图已保存: {filepath}")
return True
except Exception as e:
self.log(f"截图失败: {str(e)}")
return False
def close(self):
"""完全关闭浏览器进程(每个账号独立)并确保资源释放"""
errors = []
# 第一步:关闭上下文
if self.context:
try:
self.context.close()
self.log("上下文已关闭")
except Exception as e:
error_msg = f"关闭上下文时出错: {str(e)}"
self.log(error_msg)
errors.append(error_msg)
# 第二步:关闭浏览器进程
if self.browser:
try:
self.browser.close()
self.log("浏览器进程已关闭")
except Exception as e:
error_msg = f"关闭浏览器时出错: {str(e)}"
self.log(error_msg)
errors.append(error_msg)
# 第三步:停止Playwright
if self.playwright:
try:
self.playwright.stop()
self.log("Playwright已停止")
except Exception as e:
error_msg = f"停止Playwright时出错: {str(e)}"
self.log(error_msg)
errors.append(error_msg)
# 第四步:清空引用,确保垃圾回收
self.context = None
self.page = None
self.main_page = None
self.browser = None
self.playwright = None
# 第五步:强制等待,确保进程完全退出
time.sleep(0.5)
if errors:
self.log(f"资源清理完成,但有{len(errors)}个警告")
else:
self.log("资源清理完成")
# 简单的测试函数
if __name__ == "__main__":
print("Playwright自动化核心 - 测试")
print("="*60)
# 创建浏览器管理器
manager = PlaywrightBrowserManager(headless=True)
try:
# 初始化浏览器
manager.initialize()
# 创建自动化实例
automation = PlaywrightAutomation(manager, "test_account")
# 登录
if automation.login("19174616018", "aa123456"):
# 浏览内容
result = automation.browse_content(
browse_type="应读",
auto_next_page=True,
auto_view_attachments=True,
interval=2.0 # 增加间隔时间
)
print(f"\n浏览结果: {result}")
# 关闭
automation.close()
finally:
# 关闭浏览器管理器
manager.close()
print("="*60)
print("测试完成")

11
requirements.txt Normal file
View File

@@ -0,0 +1,11 @@
flask==3.0.0
flask-socketio==5.3.5
flask-login==0.6.3
python-socketio==5.10.0
playwright==1.40.0
eventlet==0.33.3
schedule==1.2.0
psutil==5.9.6
pytz==2024.1
bcrypt==4.0.1
requests==2.31.0

7
static/js/socket.io.min.js vendored Normal file

File diff suppressed because one or more lines are too long

1744
templates/admin.html Normal file

File diff suppressed because it is too large Load Diff

281
templates/admin_login.html Normal file
View File

@@ -0,0 +1,281 @@
<!DOCTYPE html>
<html lang="zh-CN">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>后台管理登录 - 知识管理平台</title>
<style>
* {
margin: 0;
padding: 0;
box-sizing: border-box;
}
body {
font-family: 'Microsoft YaHei', Arial, sans-serif;
background: linear-gradient(135deg, #f093fb 0%, #f5576c 100%);
min-height: 100vh;
display: flex;
justify-content: center;
align-items: center;
}
.login-container {
background: white;
border-radius: 10px;
box-shadow: 0 10px 40px rgba(0,0,0,0.2);
width: 400px;
padding: 40px;
}
.login-header {
text-align: center;
margin-bottom: 30px;
}
.login-header h1 {
font-size: 28px;
color: #333;
margin-bottom: 10px;
}
.login-header p {
color: #666;
font-size: 14px;
}
.admin-badge {
display: inline-block;
background: linear-gradient(135deg, #f093fb 0%, #f5576c 100%);
color: white;
padding: 5px 15px;
border-radius: 20px;
font-size: 12px;
margin-bottom: 15px;
}
.form-group {
margin-bottom: 20px;
}
.form-group label {
display: block;
margin-bottom: 8px;
color: #333;
font-weight: bold;
}
.form-group input {
width: 100%;
padding: 12px;
border: 1px solid #ddd;
border-radius: 5px;
font-size: 14px;
transition: border-color 0.3s;
}
.form-group input:focus {
outline: none;
border-color: #f5576c;
}
.btn-login {
width: 100%;
padding: 12px;
background: linear-gradient(135deg, #f093fb 0%, #f5576c 100%);
color: white;
border: none;
border-radius: 5px;
font-size: 16px;
font-weight: bold;
cursor: pointer;
transition: transform 0.2s;
}
.btn-login:hover {
transform: translateY(-2px);
}
.btn-login:active {
transform: translateY(0);
}
.back-link {
text-align: center;
margin-top: 20px;
color: #666;
}
.back-link a {
color: #f5576c;
text-decoration: none;
font-weight: bold;
}
.back-link a:hover {
text-decoration: underline;
}
.error-message {
background: #ffe6e6;
color: #d63031;
padding: 10px;
border-radius: 5px;
margin-bottom: 20px;
display: none;
}
.success-message {
background: #e6ffe6;
color: #27ae60;
padding: 10px;
border-radius: 5px;
margin-bottom: 20px;
display: none;
}
.warning-box {
background: #fff3cd;
border: 1px solid #ffc107;
color: #856404;
padding: 10px;
border-radius: 5px;
margin-bottom: 20px;
font-size: 13px;
}
</style>
</head>
<body>
<div class="login-container">
<div class="login-header">
<span class="admin-badge">管理员登录</span>
<h1>后台管理系统</h1>
<p>知识管理平台</p>
</div>
<div id="errorMessage" class="error-message"></div>
<div id="successMessage" class="success-message"></div>
<form id="loginForm" onsubmit="handleLogin(event)">
<div class="form-group">
<label for="username">管理员账号</label>
<input type="text" id="username" name="username" required>
</div>
<div class="form-group">
<label for="password">密码</label>
<input type="password" id="password" name="password" required>
</div>
<div id="captchaGroup" class="form-group" style="display: none;">
<label for="captcha">验证码</label>
<div style="display: flex; gap: 10px; align-items: center;">
<input type="text" id="captcha" name="captcha" placeholder="请输入验证码" style="flex: 1;">
<span id="captchaCode" style="font-size: 20px; font-weight: bold; letter-spacing: 5px; color: #f5576c; user-select: none;">----</span>
<button type="button" onclick="refreshCaptcha()" style="padding: 8px 15px; background: #f0f0f0; border: 1px solid #ddd; border-radius: 4px; cursor: pointer;">刷新</button>
</div>
</div>
<button type="submit" class="btn-login">登录后台</button>
</form>
<div class="back-link">
<a href="/">返回用户登录</a>
</div>
</div>
<script>
let captchaSession = '';
let needCaptcha = false;
async function handleLogin(event) {
event.preventDefault();
const username = document.getElementById('username').value.trim();
const password = document.getElementById('password').value.trim();
const captchaInput = document.getElementById('captcha');
const captcha = captchaInput ? captchaInput.value.trim() : '';
const errorDiv = document.getElementById('errorMessage');
const successDiv = document.getElementById('successMessage');
errorDiv.style.display = 'none';
successDiv.style.display = 'none';
if (!username || !password) {
errorDiv.textContent = '用户名和密码不能为空';
errorDiv.style.display = 'block';
return;
}
if (needCaptcha && !captcha) {
errorDiv.textContent = '请输入验证码';
errorDiv.style.display = 'block';
return;
}
try {
const response = await fetch('/yuyx/api/login', {
method: 'POST',
headers: {
'Content-Type': 'application/json'
},
body: JSON.stringify({
username: username,
password: password,
captcha_session: captchaSession,
captcha: captcha,
need_captcha: needCaptcha
})
});
const data = await response.json();
if (response.ok) {
successDiv.textContent = '登录成功,正在跳转...';
successDiv.style.display = 'block';
setTimeout(() => {
window.location.href = '/yuyx/admin';
}, 500);
} else {
errorDiv.textContent = data.error || '登录失败';
errorDiv.style.display = 'block';
if (data.need_captcha) {
needCaptcha = true;
document.getElementById('captchaGroup').style.display = 'block';
await generateCaptcha();
}
}
} catch (error) {
errorDiv.textContent = '网络错误,请稍后重试';
errorDiv.style.display = 'block';
}
}
async function generateCaptcha() {
try {
const response = await fetch('/api/generate_captcha', {
method: 'POST',
headers: {
'Content-Type': 'application/json'
}
});
const data = await response.json();
if (data.session_id && data.captcha) {
captchaSession = data.session_id;
document.getElementById('captchaCode').textContent = data.captcha;
}
} catch (error) {
console.error('生成验证码失败:', error);
}
}
async function refreshCaptcha() {
await generateCaptcha();
document.getElementById('captcha').value = '';
}
</script>
</body>
</html>

2334
templates/index.html Normal file

File diff suppressed because it is too large Load Diff

480
templates/login.html Normal file
View File

@@ -0,0 +1,480 @@
<!DOCTYPE html>
<html lang="zh-CN">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>用户登录 - 知识管理平台</title>
<style>
* {
margin: 0;
padding: 0;
box-sizing: border-box;
}
body {
font-family: 'Microsoft YaHei', Arial, sans-serif;
background: linear-gradient(135deg, #56CCF2 0%, #2F80ED 100%);
min-height: 100vh;
display: flex;
justify-content: center;
align-items: center;
}
.login-container {
background: white;
border-radius: 10px;
box-shadow: 0 10px 40px rgba(0,0,0,0.2);
width: 400px;
padding: 40px;
}
.login-header {
text-align: center;
margin-bottom: 30px;
}
.login-header h1 {
font-size: 28px;
color: #333;
margin-bottom: 10px;
}
.login-header p {
color: #666;
font-size: 14px;
}
.form-group {
margin-bottom: 20px;
}
.form-group label {
display: block;
margin-bottom: 8px;
color: #333;
font-weight: bold;
}
.form-group input {
width: 100%;
padding: 12px;
border: 1px solid #ddd;
border-radius: 5px;
font-size: 14px;
transition: border-color 0.3s;
}
.form-group input:focus {
outline: none;
border-color: #2F80ED;
}
.btn-login {
width: 100%;
padding: 12px;
background: linear-gradient(135deg, #56CCF2 0%, #2F80ED 100%);
color: white;
border: none;
border-radius: 5px;
font-size: 16px;
font-weight: bold;
cursor: pointer;
transition: transform 0.2s;
}
.btn-login:hover {
transform: translateY(-2px);
}
.btn-login:active {
transform: translateY(0);
}
.register-link {
text-align: center;
margin-top: 20px;
color: #666;
}
.register-link a {
color: #2F80ED;
text-decoration: none;
font-weight: bold;
}
.register-link a:hover {
text-decoration: underline;
}
.error-message {
background: #ffe6e6;
color: #d63031;
padding: 10px;
border-radius: 5px;
margin-bottom: 20px;
display: none;
}
.success-message {
background: #e6ffe6;
color: #27ae60;
padding: 10px;
border-radius: 5px;
margin-bottom: 20px;
display: none;
}
.forgot-password-link {
text-align: right;
margin-top: -10px;
margin-bottom: 20px;
}
.forgot-password-link a {
color: #2F80ED;
text-decoration: none;
font-size: 14px;
}
.forgot-password-link a:hover {
text-decoration: underline;
}
.modal {
display: none;
position: fixed;
top: 0;
left: 0;
width: 100%;
height: 100%;
background: rgba(0, 0, 0, 0.5);
justify-content: center;
align-items: center;
z-index: 1000;
}
.modal.active {
display: flex;
}
.modal-content {
background: white;
border-radius: 10px;
padding: 30px;
width: 90%;
max-width: 400px;
box-shadow: 0 10px 40px rgba(0,0,0,0.3);
}
.modal-header {
margin-bottom: 20px;
}
.modal-header h2 {
font-size: 22px;
color: #333;
margin-bottom: 10px;
}
.modal-header p {
font-size: 13px;
color: #666;
}
.modal-footer {
display: flex;
gap: 10px;
margin-top: 20px;
}
.btn-secondary {
flex: 1;
padding: 12px;
background: #6c757d;
color: white;
border: none;
border-radius: 5px;
font-size: 14px;
font-weight: bold;
cursor: pointer;
transition: transform 0.2s;
}
.btn-secondary:hover {
transform: translateY(-2px);
}
.btn-primary {
flex: 1;
padding: 12px;
background: linear-gradient(135deg, #56CCF2 0%, #2F80ED 100%);
color: white;
border: none;
border-radius: 5px;
font-size: 14px;
font-weight: bold;
cursor: pointer;
transition: transform 0.2s;
}
.btn-primary:hover {
transform: translateY(-2px);
}
</style>
</head>
<body>
<div class="login-container">
<div class="login-header">
<h1>用户登录</h1>
</div>
<div id="errorMessage" class="error-message"></div>
<div id="successMessage" class="success-message"></div>
<form id="loginForm" onsubmit="handleLogin(event)">
<div class="form-group">
<label for="username">用户名</label>
<input type="text" id="username" name="username" required>
</div>
<div class="form-group">
<label for="password">密码</label>
<input type="password" id="password" name="password" required>
</div>
<!-- 验证码区域(第一次失败后显示) -->
<div id="captchaGroup" class="form-group" style="display: none;">
<label for="captcha">验证码</label>
<div style="display: flex; gap: 10px; align-items: center;">
<input type="text" id="captcha" name="captcha" placeholder="请输入验证码" style="flex: 1;">
<span id="captchaCode" style="font-size: 20px; font-weight: bold; letter-spacing: 5px; color: #4CAF50; user-select: none;">----</span>
<button type="button" onclick="refreshCaptcha()" style="padding: 8px 15px; background: #f0f0f0; border: 1px solid #ddd; border-radius: 4px; cursor: pointer;">刷新</button>
</div>
</div>
<div class="forgot-password-link">
<a href="#" onclick="showForgotPassword(event)">忘记密码?</a>
</div>
<button type="submit" class="btn-login">登录</button>
</form>
<div class="register-link">
还没有账号? <a href="/register">立即注册</a>
</div>
</div>
<!-- 忘记密码弹窗 -->
<div id="forgotPasswordModal" class="modal" onclick="closeModal(event)">
<div class="modal-content">
<div class="modal-header">
<h2>重置密码</h2>
<p>请填写您的用户名和新密码,管理员审核通过后生效</p>
</div>
<div id="modalErrorMessage" class="error-message"></div>
<div id="modalSuccessMessage" class="success-message"></div>
<form id="resetPasswordForm" onsubmit="handleResetPassword(event)">
<div class="form-group">
<label for="resetUsername">用户名</label>
<input type="text" id="resetUsername" required>
</div>
<div class="form-group">
<label for="resetEmail">邮箱(可选)</label>
<input type="email" id="resetEmail" placeholder="用于验证身份">
</div>
<div class="form-group">
<label for="resetNewPassword">新密码</label>
<input type="password" id="resetNewPassword" required>
</div>
<div class="modal-footer">
<button type="button" class="btn-secondary" onclick="closeForgotPassword()">取消</button>
<button type="submit" class="btn-primary">提交申请</button>
</div>
</form>
</div>
</div>
<script>
// 全局变量存储验证码session
let captchaSession = '';
let needCaptcha = false;
async function handleLogin(event) {
event.preventDefault();
const username = document.getElementById('username').value.trim();
const password = document.getElementById('password').value.trim();
const captchaInput = document.getElementById('captcha');
const captcha = captchaInput ? captchaInput.value.trim() : '';
const errorDiv = document.getElementById('errorMessage');
const successDiv = document.getElementById('successMessage');
errorDiv.style.display = 'none';
successDiv.style.display = 'none';
if (!username || !password) {
errorDiv.textContent = '用户名和密码不能为空';
errorDiv.style.display = 'block';
return;
}
// 如果需要验证码但没有输入
if (needCaptcha && !captcha) {
errorDiv.textContent = '请输入验证码';
errorDiv.style.display = 'block';
return;
}
try {
const response = await fetch('/api/login', {
method: 'POST',
headers: {
'Content-Type': 'application/json'
},
body: JSON.stringify({
username,
password,
captcha_session: captchaSession,
captcha: captcha,
need_captcha: needCaptcha
})
});
const data = await response.json();
if (response.ok) {
successDiv.textContent = '登录成功,正在跳转...';
successDiv.style.display = 'block';
setTimeout(() => {
window.location.href = '/app';
}, 500);
} else {
// 显示详细错误信息
errorDiv.textContent = data.error || '登录失败';
errorDiv.style.display = 'block';
// 如果返回需要验证码,显示验证码区域并生成验证码
if (data.need_captcha) {
needCaptcha = true;
document.getElementById('captchaGroup').style.display = 'block';
await generateCaptcha();
}
}
} catch (error) {
errorDiv.textContent = '网络错误,请稍后重试';
errorDiv.style.display = 'block';
}
}
// 忘记密码功能
function showForgotPassword(event) {
event.preventDefault();
document.getElementById('forgotPasswordModal').classList.add('active');
document.body.style.overflow = 'hidden';
}
function closeForgotPassword() {
document.getElementById('forgotPasswordModal').classList.remove('active');
document.body.style.overflow = '';
// 清空表单
document.getElementById('resetPasswordForm').reset();
document.getElementById('modalErrorMessage').style.display = 'none';
document.getElementById('modalSuccessMessage').style.display = 'none';
}
function closeModal(event) {
if (event.target.id === 'forgotPasswordModal') {
closeForgotPassword();
}
}
async function handleResetPassword(event) {
event.preventDefault();
const username = document.getElementById('resetUsername').value.trim();
const email = document.getElementById('resetEmail').value.trim();
const newPassword = document.getElementById('resetNewPassword').value.trim();
const modalErrorDiv = document.getElementById('modalErrorMessage');
const modalSuccessDiv = document.getElementById('modalSuccessMessage');
modalErrorDiv.style.display = 'none';
modalSuccessDiv.style.display = 'none';
if (!username || !newPassword) {
modalErrorDiv.textContent = '用户名和新密码不能为空';
modalErrorDiv.style.display = 'block';
return;
}
if (newPassword.length < 6) {
modalErrorDiv.textContent = '密码长度至少6位';
modalErrorDiv.style.display = 'block';
return;
}
try {
const response = await fetch('/api/reset_password_request', {
method: 'POST',
headers: {
'Content-Type': 'application/json'
},
body: JSON.stringify({ username, email, new_password: newPassword })
});
const data = await response.json();
if (response.ok) {
modalSuccessDiv.textContent = '密码重置申请已提交,请等待管理员审核';
modalSuccessDiv.style.display = 'block';
setTimeout(() => {
closeForgotPassword();
}, 2000);
} else {
modalErrorDiv.textContent = data.error || '申请失败';
modalErrorDiv.style.display = 'block';
}
} catch (error) {
modalErrorDiv.textContent = '网络错误,请稍后重试';
modalErrorDiv.style.display = 'block';
}
}
// ESC键关闭弹窗
document.addEventListener('keydown', (event) => {
if (event.key === 'Escape') {
closeForgotPassword();
}
});
// 生成验证码
async function generateCaptcha() {
try {
const response = await fetch('/api/generate_captcha', {
method: 'POST',
headers: {
'Content-Type': 'application/json'
}
});
const data = await response.json();
if (data.session_id && data.captcha) {
captchaSession = data.session_id;
document.getElementById('captchaCode').textContent = data.captcha;
}
} catch (error) {
console.error('生成验证码失败:', error);
}
}
// 刷新验证码
async function refreshCaptcha() {
await generateCaptcha();
document.getElementById('captcha').value = '';
}
</script>
</body>
</html>

258
templates/register.html Normal file
View File

@@ -0,0 +1,258 @@
<!DOCTYPE html>
<html lang="zh-CN">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>用户注册 - 知识管理平台</title>
<style>
* {
margin: 0;
padding: 0;
box-sizing: border-box;
}
body {
font-family: 'Microsoft YaHei', Arial, sans-serif;
background: linear-gradient(135deg, #56CCF2 0%, #2F80ED 100%);
min-height: 100vh;
display: flex;
justify-content: center;
align-items: center;
}
.register-container {
background: white;
border-radius: 10px;
box-shadow: 0 10px 40px rgba(0,0,0,0.2);
width: 400px;
padding: 40px;
}
.register-header {
text-align: center;
margin-bottom: 30px;
}
.register-header h1 {
font-size: 28px;
color: #333;
margin-bottom: 10px;
}
.register-header p {
color: #666;
font-size: 14px;
}
.form-group {
margin-bottom: 20px;
}
.form-group label {
display: block;
margin-bottom: 8px;
color: #333;
font-weight: bold;
}
.form-group input {
width: 100%;
padding: 12px;
border: 1px solid #ddd;
border-radius: 5px;
font-size: 14px;
transition: border-color 0.3s;
}
.form-group input:focus {
outline: none;
border-color: #2F80ED;
}
.form-group small {
color: #888;
font-size: 12px;
display: block;
margin-top: 5px;
}
.btn-register {
width: 100%;
padding: 12px;
background: linear-gradient(135deg, #56CCF2 0%, #2F80ED 100%);
color: white;
border: none;
border-radius: 5px;
font-size: 16px;
font-weight: bold;
cursor: pointer;
transition: transform 0.2s;
}
.btn-register:hover {
transform: translateY(-2px);
}
.btn-register:active {
transform: translateY(0);
}
.login-link {
text-align: center;
margin-top: 20px;
color: #666;
}
.login-link a {
color: #2F80ED;
text-decoration: none;
font-weight: bold;
}
.login-link a:hover {
text-decoration: underline;
}
.error-message {
background: #ffe6e6;
color: #d63031;
padding: 10px;
border-radius: 5px;
margin-bottom: 20px;
display: none;
}
.success-message {
background: #e6ffe6;
color: #27ae60;
padding: 10px;
border-radius: 5px;
margin-bottom: 20px;
display: none;
}
</style>
</head>
<body>
<div class="register-container">
<div class="register-header">
<h1>用户注册</h1>
</div>
<div id="errorMessage" class="error-message"></div>
<div id="successMessage" class="success-message"></div>
<form id="registerForm" onsubmit="handleRegister(event)">
<div class="form-group">
<label for="username">用户名 *</label>
<input type="text" id="username" name="username" required minlength="3">
<small>至少3个字符</small>
</div>
<div class="form-group">
<label for="password">密码 *</label>
<input type="password" id="password" name="password" required minlength="6">
<small>至少6个字符</small>
</div>
<div class="form-group">
<label for="confirm_password">确认密码 *</label>
<input type="password" id="confirm_password" name="confirm_password" required>
</div>
<div class="form-group">
<label for="email">邮箱</label>
<input type="email" id="email" name="email">
<small>选填,用于接收审核通知</small>
</div>
<div class="form-group">
<label for="captcha">验证码</label>
<div style="display: flex; gap: 10px; align-items: center;">
<input type="text" id="captcha" placeholder="请输入验证码" required style="flex: 1;">
<span id="captchaCode" style="font-size: 20px; font-weight: bold; letter-spacing: 5px; color: #4CAF50;">----</span>
<button type="button" onclick="refreshCaptcha()" style="padding: 8px 15px; background: #f0f0f0; border: 1px solid #ddd; border-radius: 4px; cursor: pointer;">刷新</button>
</div>
</div>
<button type="submit" class="btn-register">注册</button>
</form>
<div class="login-link">
已有账号? <a href="/login">立即登录</a>
</div>
</div>
<script>
let captchaSession = '';
window.onload = function() { generateCaptcha(); };
async function handleRegister(event) {
event.preventDefault();
const username = document.getElementById('username').value.trim();
const password = document.getElementById('password').value.trim();
const confirmPassword = document.getElementById('confirm_password').value.trim();
const email = document.getElementById('email').value.trim();
const errorDiv = document.getElementById('errorMessage');
const successDiv = document.getElementById('successMessage');
errorDiv.style.display = 'none';
successDiv.style.display = 'none';
// 验证
if (username.length < 3) {
errorDiv.textContent = '用户名至少3个字符';
errorDiv.style.display = 'block';
return;
}
if (password.length < 6) {
errorDiv.textContent = '密码至少6个字符';
errorDiv.style.display = 'block';
return;
}
if (password !== confirmPassword) {
errorDiv.textContent = '两次输入的密码不一致';
errorDiv.style.display = 'block';
return;
}
try {
const response = await fetch('/api/register', {
method: 'POST',
headers: {
'Content-Type': 'application/json'
},
body: JSON.stringify({ username, password, email, captcha_session: captchaSession, captcha: document.getElementById('captcha').value.trim() })
});
const data = await response.json();
if (response.ok) {
successDiv.textContent = data.message || '注册成功,请等待管理员审核';
successDiv.style.display = 'block';
// 清空表单
document.getElementById('registerForm').reset();
// 3秒后跳转到登录页
setTimeout(() => {
window.location.href = '/login';
}, 3000);
} else {
errorDiv.textContent = data.error || '注册失败';
errorDiv.style.display = 'block';
}
} catch (error) {
errorDiv.textContent = '网络错误,请稍后重试';
errorDiv.style.display = 'block';
}
}
async function generateCaptcha() {
const resp = await fetch('/api/generate_captcha', {method: 'POST', headers: {'Content-Type': 'application/json'}});
const data = await resp.json();
if (data.session_id && data.captcha) { captchaSession = data.session_id; document.getElementById('captchaCode').textContent = data.captcha; }
}
async function refreshCaptcha() { await generateCaptcha(); document.getElementById('captcha').value = ''; }
</script>
</body>
</html>