新增项目README文件
This commit is contained in:
zyj
420
README.md
420
README.md
@@ -1,59 +1,409 @@
|
|||||||
# Welcome to Your New Wails3 Project!
|
# Go Site Clone
|
||||||
|
|
||||||
Congratulations on generating your Wails3 application! This README will guide you through the next steps to get your project up and running.
|
<div align="center">
|
||||||
|
|
||||||
## Getting Started
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
1. Navigate to your project directory in the terminal.
|
A powerful website cloning and local deployment tool built with Wails3 + Nuxt.js
|
||||||
|
|
||||||
2. To run your application in development mode, use the following command:
|
[Features](#-features) • [Quick Start](#-quick-start) • [Usage](#-usage) • [Architecture](#-architecture) • [Configuration](#-configuration)
|
||||||
|
|
||||||
```
|
</div>
|
||||||
wails3 dev
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📖 Overview
|
||||||
|
|
||||||
|
Go Site Clone is a powerful website cloning tool that can completely capture all resources of a website (HTML, CSS, JavaScript, images, videos, etc.) and support local deployment. It uses a real browser engine for resource capture, equipped with comprehensive anti-detection mechanisms and browser fingerprint spoofing system.
|
||||||
|
|
||||||
|
### ✨ Highlights
|
||||||
|
|
||||||
|
- 🚀 **Real Browser Engine** - Based on Chrome DevTools Protocol, simulating real user behavior
|
||||||
|
- 🎭 **Fingerprint Spoofing** - Canvas fingerprint, timezone, language, UserAgent comprehensive disguise
|
||||||
|
- 🔄 **Smart Resource Management** - Auto deduplication, same-domain filtering, categorized storage
|
||||||
|
- 📦 **Complete Site Cloning** - Support for static resources and dynamic content
|
||||||
|
- 🖥️ **Cross-Platform** - One-click packaging for Windows, macOS, Linux
|
||||||
|
- 💫 **Modern UI** - Nuxt.js + Ant Design Vue responsive interface
|
||||||
|
- 🌐 **Local Site Service** - Integrated Nginx for one-click local mirror site startup
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🎯 Features
|
||||||
|
|
||||||
|
### Resource Capture
|
||||||
|
|
||||||
|
- ✅ Complete HTML page cloning
|
||||||
|
- ✅ Batch CSS stylesheet download
|
||||||
|
- ✅ JavaScript file capture
|
||||||
|
- ✅ Image resources (JPG, PNG, GIF, SVG, WebP)
|
||||||
|
- ✅ Video files (MP4, WebM, etc.)
|
||||||
|
- ✅ Fonts and other static resources
|
||||||
|
- ✅ Smart link traversal (auto-track internal links)
|
||||||
|
|
||||||
|
### Anti-Detection
|
||||||
|
|
||||||
|
- 🎭 Canvas fingerprint randomization
|
||||||
|
- 🌍 Timezone simulation (global timezone support)
|
||||||
|
- 🗣️ Language and locale spoofing
|
||||||
|
- 🔐 Custom UserAgent
|
||||||
|
- 📍 Geolocation coordinate spoofing
|
||||||
|
- 🕵️ Stealth plugin integration (bypass anti-bot detection)
|
||||||
|
|
||||||
|
### File Management
|
||||||
|
|
||||||
|
- 📁 Auto-categorized storage by domain
|
||||||
|
- 🔍 Resource type classification
|
||||||
|
- 📊 Real-time download progress
|
||||||
|
- 🗂️ Local site list view
|
||||||
|
- 📂 One-click folder opening
|
||||||
|
|
||||||
|
### Local Deployment
|
||||||
|
|
||||||
|
- 🌐 Integrated Nginx server
|
||||||
|
- ⚡ Quick start local mirror sites
|
||||||
|
- 🔧 Custom port configuration
|
||||||
|
- 📝 Site configuration management
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🚀 Quick Start
|
||||||
|
|
||||||
|
### Requirements
|
||||||
|
|
||||||
|
- Go 1.24 or higher
|
||||||
|
- Node.js 18+ and npm/pnpm
|
||||||
|
- Wails CLI v3
|
||||||
|
|
||||||
|
### Installation
|
||||||
|
|
||||||
|
#### 1. Install Wails CLI
|
||||||
|
|
||||||
|
```bash
|
||||||
|
go install github.com/wailsapp/wails/v3/cmd/wails3@latest
|
||||||
|
```
|
||||||
|
|
||||||
|
#### 2. Clone Repository
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git clone https://github.com/zhuy1228/go-site-clone.git
|
||||||
|
cd go-site-clone
|
||||||
|
```
|
||||||
|
|
||||||
|
#### 3. Install Go Dependencies
|
||||||
|
|
||||||
|
```bash
|
||||||
|
go mod download
|
||||||
|
```
|
||||||
|
|
||||||
|
#### 4. Install Frontend Dependencies
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd frontend
|
||||||
|
npm install
|
||||||
|
# or use pnpm
|
||||||
|
pnpm install
|
||||||
|
```
|
||||||
|
|
||||||
|
### Development Mode
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Start dev server with hot-reload
|
||||||
|
wails3 dev
|
||||||
|
```
|
||||||
|
|
||||||
|
The app will start in development mode with auto-reload for both frontend and backend changes.
|
||||||
|
|
||||||
|
### Production Build
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Build production version
|
||||||
|
wails3 build
|
||||||
|
|
||||||
|
# Output in build/ directory
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📚 Usage
|
||||||
|
|
||||||
|
### Basic Workflow
|
||||||
|
|
||||||
|
1. **Start Application**
|
||||||
|
```bash
|
||||||
|
wails3 dev # or run the built executable
|
||||||
```
|
```
|
||||||
|
|
||||||
This will start your application and enable hot-reloading for both frontend and backend changes.
|
2. **Enter Target Website URL**
|
||||||
|
- Input the website URL you want to clone
|
||||||
|
- Example: `https://example.com`
|
||||||
|
|
||||||
3. To build your application for production, use:
|
3. **Get Resource List**
|
||||||
|
- Click "Get Resources" button
|
||||||
|
- System automatically analyzes and lists all resources
|
||||||
|
|
||||||
```
|
4. **Download Site**
|
||||||
wails3 build
|
- Select resource types to download
|
||||||
```
|
- Click "Download" to start cloning
|
||||||
|
- View real-time download progress
|
||||||
|
|
||||||
This will create a production-ready executable in the `build` directory.
|
5. **Local Preview**
|
||||||
|
- Select a site from download list
|
||||||
|
- Click "Open Folder" to view files
|
||||||
|
- Or use integrated Nginx service for local deployment
|
||||||
|
|
||||||
## Exploring Wails3 Features
|
### Configuration
|
||||||
|
|
||||||
Now that you have your project set up, it's time to explore the features that Wails3 offers:
|
Edit `config.yaml` to customize settings:
|
||||||
|
|
||||||
1. **Check out the examples**: The best way to learn is by example. Visit the `examples` directory in the `v3/examples` directory to see various sample applications.
|
```yaml
|
||||||
|
# Application name
|
||||||
|
appName: "Go Site Clone"
|
||||||
|
|
||||||
2. **Run an example**: To run any of the examples, navigate to the example's directory and use:
|
# Local service port
|
||||||
|
port: 6997
|
||||||
|
|
||||||
```
|
# WebSocket connection URL
|
||||||
go run .
|
wsUrl: "106.12.33.188:6996"
|
||||||
```
|
|
||||||
|
|
||||||
Note: Some examples may be under development during the alpha phase.
|
# STUN server address
|
||||||
|
stunUrl: "stun:106.12.33.188:3478"
|
||||||
|
|
||||||
3. **Explore the documentation**: Visit the [Wails3 documentation](https://v3.wails.io/) for in-depth guides and API references.
|
# API service URL
|
||||||
|
apiUrl: "http://106.12.33.188:6996"
|
||||||
|
|
||||||
4. **Join the community**: Have questions or want to share your progress? Join the [Wails Discord](https://discord.gg/JDdSxwjhGf) or visit the [Wails discussions on GitHub](https://github.com/wailsapp/wails/discussions).
|
# Website file save directory
|
||||||
|
siteFileDir: "www"
|
||||||
|
```
|
||||||
|
|
||||||
## Project Structure
|
### API Methods
|
||||||
|
|
||||||
Take a moment to familiarize yourself with your project structure:
|
The application exposes the following methods for frontend:
|
||||||
|
|
||||||
- `frontend/`: Contains your frontend code (HTML, CSS, JavaScript/TypeScript)
|
#### `GetResources(url string)`
|
||||||
- `main.go`: The entry point of your Go backend
|
Get all resource lists for specified URL
|
||||||
- `app.go`: Define your application structure and methods here
|
|
||||||
- `wails.json`: Configuration file for your Wails project
|
|
||||||
|
|
||||||
## Next Steps
|
**Parameters:**
|
||||||
|
- `url`: Target website URL
|
||||||
|
|
||||||
1. Modify the frontend in the `frontend/` directory to create your desired UI.
|
**Returns:** `ResourcesList` object with categorized resources
|
||||||
2. Add backend functionality in `main.go`.
|
|
||||||
3. Use `wails3 dev` to see your changes in real-time.
|
|
||||||
4. When ready, build your application with `wails3 build`.
|
|
||||||
|
|
||||||
Happy coding with Wails3! If you encounter any issues or have questions, don't hesitate to consult the documentation or reach out to the Wails community.
|
#### `DownloadSite(uri string, resources ResourcesList)`
|
||||||
|
Download website resources to local
|
||||||
|
|
||||||
|
**Parameters:**
|
||||||
|
- `uri`: Website URL
|
||||||
|
- `resources`: Resource list object
|
||||||
|
|
||||||
|
**Returns:** `bool` - Success status
|
||||||
|
|
||||||
|
#### `GetDownloadList()`
|
||||||
|
Get list of downloaded websites
|
||||||
|
|
||||||
|
**Returns:** `[]FileDir` - File directory list
|
||||||
|
|
||||||
|
#### `OpenSiteFileDir(path string)`
|
||||||
|
Open folder for specified website
|
||||||
|
|
||||||
|
**Parameters:**
|
||||||
|
- `path`: Relative path
|
||||||
|
|
||||||
|
**Returns:** `bool` - Operation success status
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🏗️ Architecture
|
||||||
|
|
||||||
|
### Tech Stack
|
||||||
|
|
||||||
|
**Backend (Go)**
|
||||||
|
- [Wails v3](https://wails.io/) - Application framework
|
||||||
|
- [go-rod](https://github.com/go-rod/rod) - Browser automation
|
||||||
|
- [go-rod/stealth](https://github.com/go-rod/stealth) - Anti-detection
|
||||||
|
- [goquery](https://github.com/PuerkitoBio/goquery) - HTML parsing
|
||||||
|
- [go-sqlite3](https://github.com/mattn/go-sqlite3) - Database
|
||||||
|
- [yaml.v3](https://github.com/go-yaml/yaml) - Configuration
|
||||||
|
|
||||||
|
**Frontend (Nuxt.js)**
|
||||||
|
- [Nuxt 4](https://nuxt.com/) - Vue framework
|
||||||
|
- [Vue 3](https://vuejs.org/) - Reactive framework
|
||||||
|
- [Ant Design Vue](https://antdv.com/) - UI component library
|
||||||
|
- [Chart.js](https://www.chartjs.org/) - Charts
|
||||||
|
- [Wails Runtime](https://wails.io/) - Frontend-backend communication
|
||||||
|
|
||||||
|
### Project Structure
|
||||||
|
|
||||||
|
```
|
||||||
|
go-site-clone/
|
||||||
|
├── app.go # Application main logic
|
||||||
|
├── main.go # Program entry
|
||||||
|
├── config.yaml # Configuration file
|
||||||
|
├── go.mod # Go module dependencies
|
||||||
|
│
|
||||||
|
├── config/ # Configuration loader
|
||||||
|
│ └── index.go
|
||||||
|
│
|
||||||
|
├── services/ # Business services
|
||||||
|
│ ├── site_service.go # Website cloning core
|
||||||
|
│ └── nginx_service.go # Nginx service management
|
||||||
|
│
|
||||||
|
├── libs/ # Core libraries
|
||||||
|
│ ├── browser.go # Browser pool management
|
||||||
|
│ ├── chrome.go # Chrome instance creation
|
||||||
|
│ ├── sqlite.go # Database operations
|
||||||
|
│ ├── pack.go # Packaging utilities
|
||||||
|
│ └── browser-fingerprint/ # Browser fingerprint spoofing
|
||||||
|
│ ├── index.go
|
||||||
|
│ ├── canvas.go # Canvas fingerprint
|
||||||
|
│ └── timezone.go # Timezone simulation
|
||||||
|
│
|
||||||
|
├── types/ # Type definitions
|
||||||
|
│ ├── fingerprint_type.go # Fingerprint parameter types
|
||||||
|
│ ├── task_type.go # Task types
|
||||||
|
│ └── nginx_template.go # Nginx templates
|
||||||
|
│
|
||||||
|
├── utils/ # Utility functions
|
||||||
|
│ ├── file.go # File download/management
|
||||||
|
│ └── index.go # Environment checks
|
||||||
|
│
|
||||||
|
├── frontend/ # Nuxt.js frontend
|
||||||
|
│ ├── nuxt.config.ts # Nuxt configuration
|
||||||
|
│ ├── package.json # Frontend dependencies
|
||||||
|
│ ├── app/ # Application code
|
||||||
|
│ │ ├── app.vue # Root component
|
||||||
|
│ │ ├── components/ # Vue components
|
||||||
|
│ │ ├── pages/ # Page routes
|
||||||
|
│ │ ├── layouts/ # Layout templates
|
||||||
|
│ │ └── plugins/ # Plugins
|
||||||
|
│ └── bindings/ # Go binding types
|
||||||
|
│
|
||||||
|
├── user-data/ # Browser user data
|
||||||
|
├── www/ # Website file storage
|
||||||
|
├── plugin/ # Plugins
|
||||||
|
│ └── nginx/ # Nginx plugin
|
||||||
|
└── build/ # Build configuration
|
||||||
|
├── darwin/ # macOS packaging config
|
||||||
|
├── linux/ # Linux packaging config
|
||||||
|
└── windows/ # Windows packaging config
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ⚙️ Configuration
|
||||||
|
|
||||||
|
### Browser Fingerprint Config
|
||||||
|
|
||||||
|
```go
|
||||||
|
type BrowserFingerprintParams struct {
|
||||||
|
Canvas bool // Randomize Canvas fingerprint
|
||||||
|
TimeZone string // Timezone, e.g., "America/New_York"
|
||||||
|
Language string // Language, e.g., "en-US"
|
||||||
|
UserAgent string // Custom UserAgent
|
||||||
|
GeoLocation string // Geolocation, e.g., "40.7128,-74.0060"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔧 Development
|
||||||
|
|
||||||
|
### Adding Features
|
||||||
|
|
||||||
|
1. **Backend Services**
|
||||||
|
- Create new service in `services/`
|
||||||
|
- Implement business logic
|
||||||
|
- Expose methods in `app.go` for frontend
|
||||||
|
|
||||||
|
2. **Frontend UI**
|
||||||
|
- Add new pages in `frontend/app/pages/`
|
||||||
|
- Add components in `frontend/app/components/`
|
||||||
|
- Use Wails Runtime to call backend methods
|
||||||
|
|
||||||
|
### Debugging
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# View detailed logs
|
||||||
|
wails3 dev -v
|
||||||
|
|
||||||
|
# Build with verbose output
|
||||||
|
wails3 build -v
|
||||||
|
|
||||||
|
# Clean build cache
|
||||||
|
wails3 build -clean
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📦 Build & Deploy
|
||||||
|
|
||||||
|
### Windows
|
||||||
|
|
||||||
|
```bash
|
||||||
|
wails3 build -platform windows/amd64
|
||||||
|
```
|
||||||
|
|
||||||
|
Executable in `build/bin/`
|
||||||
|
|
||||||
|
### macOS
|
||||||
|
|
||||||
|
```bash
|
||||||
|
wails3 build -platform darwin/universal
|
||||||
|
```
|
||||||
|
|
||||||
|
Generates `.app` bundle
|
||||||
|
|
||||||
|
### Linux
|
||||||
|
|
||||||
|
```bash
|
||||||
|
wails3 build -platform linux/amd64
|
||||||
|
```
|
||||||
|
|
||||||
|
Supports `.deb`, `.rpm`, AppImage formats
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🤝 Contributing
|
||||||
|
|
||||||
|
Issues and Pull Requests are welcome!
|
||||||
|
|
||||||
|
1. Fork the repository
|
||||||
|
2. Create feature branch (`git checkout -b feature/AmazingFeature`)
|
||||||
|
3. Commit changes (`git commit -m 'Add some AmazingFeature'`)
|
||||||
|
4. Push to branch (`git push origin feature/AmazingFeature`)
|
||||||
|
5. Submit Pull Request
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📄 License
|
||||||
|
|
||||||
|
This project is licensed under the MIT License - see [LICENSE](LICENSE) file
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🙏 Acknowledgments
|
||||||
|
|
||||||
|
- [Wails](https://wails.io/) - Excellent Go + Web framework
|
||||||
|
- [go-rod](https://go-rod.github.io/) - Powerful browser automation
|
||||||
|
- [Nuxt.js](https://nuxt.com/) - Elegant Vue framework
|
||||||
|
- [Ant Design Vue](https://antdv.com/) - Beautiful UI components
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📞 Contact
|
||||||
|
|
||||||
|
- Author: zhuy1228
|
||||||
|
- Project: [https://github.com/zhuy1228/go-site-clone](https://github.com/zhuy1228/go-site-clone)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
<div align="center">
|
||||||
|
|
||||||
|
**If this project helps you, please give it a ⭐️**
|
||||||
|
|
||||||
|
Made with ❤️ by zhuy1228
|
||||||
|
|
||||||
|
**[中文文档](README_CN.md)**
|
||||||
|
|
||||||
|
</div>
|
||||||
|
|||||||
450
README_CN.md
Normal file
450
README_CN.md
Normal file
@@ -0,0 +1,450 @@
|
|||||||
|
# Go Site Clone
|
||||||
|
|
||||||
|
<div align="center">
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
基于 Wails3 + Nuxt.js 开发的网站克隆与本地部署工具
|
||||||
|
|
||||||
|
[功能特性](#-功能特性) • [快速开始](#-快速开始) • [使用说明](#-使用说明) • [技术架构](#-技术架构) • [配置说明](#-配置说明)
|
||||||
|
|
||||||
|
</div>
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📖 项目简介
|
||||||
|
|
||||||
|
Go Site Clone 是一款强大的网站克隆工具，能够完整抓取网站的所有资源（HTML、CSS、JavaScript、图片、视频等），并支持本地部署。采用真实浏览器引擎进行资源抓取，配备完善的反检测机制和浏览器指纹伪装系统。
|
||||||
|
|
||||||
|
### ✨ 核心亮点
|
||||||
|
|
||||||
|
- 🚀 **真实浏览器引擎** - 基于 Chrome DevTools Protocol，模拟真实用户行为
|
||||||
|
- 🎭 **指纹伪装系统** - Canvas 指纹、时区、语言、UserAgent 全方位伪装
|
||||||
|
- 🔄 **智能资源管理** - 自动去重、同域过滤、分类存储
|
||||||
|
- 📦 **完整站点克隆** - 支持静态资源、动态内容的完整抓取
|
||||||
|
- 🖥️ **跨平台支持** - Windows、macOS、Linux 一键打包
|
||||||
|
- 💫 **现代化界面** - Nuxt.js + Ant Design Vue 响应式 UI
|
||||||
|
- 🌐 **本地站点服务** - 集成 Nginx，一键启动本地镜像站点
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🎯 功能特性
|
||||||
|
|
||||||
|
### 资源抓取
|
||||||
|
|
||||||
|
- ✅ HTML 页面完整克隆
|
||||||
|
- ✅ CSS 样式表批量下载
|
||||||
|
- ✅ JavaScript 脚本文件抓取
|
||||||
|
- ✅ 图片资源（JPG、PNG、GIF、SVG、WebP）
|
||||||
|
- ✅ 视频文件（MP4、WebM 等）
|
||||||
|
- ✅ 字体文件和其他静态资源
|
||||||
|
- ✅ 智能链接遍历（站内链接自动跟踪）
|
||||||
|
|
||||||
|
### 反检测技术
|
||||||
|
|
||||||
|
- 🎭 Canvas 指纹随机化
|
||||||
|
- 🌍 时区模拟（支持全球时区）
|
||||||
|
- 🗣️ 语言和地域伪装
|
||||||
|
- 🔐 UserAgent 定制
|
||||||
|
- 📍 地理位置坐标伪装
|
||||||
|
- 🕵️ Stealth 插件集成（绕过反爬虫检测）
|
||||||
|
|
||||||
|
### 文件管理
|
||||||
|
|
||||||
|
- 📁 按域名自动分类存储
|
||||||
|
- 🔍 资源类型分类管理
|
||||||
|
- 📊 下载进度实时显示
|
||||||
|
- 🗂️ 本地站点列表查看
|
||||||
|
- 📂 一键打开文件夹
|
||||||
|
|
||||||
|
### 本地部署
|
||||||
|
|
||||||
|
- 🌐 集成 Nginx 服务器
|
||||||
|
- ⚡ 快速启动本地镜像站点
|
||||||
|
- 🔧 自定义端口配置
|
||||||
|
- 📝 站点配置管理
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🚀 快速开始
|
||||||
|
|
||||||
|
### 环境要求
|
||||||
|
|
||||||
|
- Go 1.24 或更高版本
|
||||||
|
- Node.js 18+ 和 npm/pnpm
|
||||||
|
- Wails CLI v3
|
||||||
|
|
||||||
|
### 安装依赖
|
||||||
|
|
||||||
|
#### 1. 安装 Wails CLI
|
||||||
|
|
||||||
|
```bash
|
||||||
|
go install github.com/wailsapp/wails/v3/cmd/wails3@latest
|
||||||
|
```
|
||||||
|
|
||||||
|
#### 2. 克隆项目
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git clone https://github.com/zhuy1228/go-site-clone.git
|
||||||
|
cd go-site-clone
|
||||||
|
```
|
||||||
|
|
||||||
|
#### 3. 安装 Go 依赖
|
||||||
|
|
||||||
|
```bash
|
||||||
|
go mod download
|
||||||
|
```
|
||||||
|
|
||||||
|
#### 4. 安装前端依赖
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd frontend
|
||||||
|
npm install
|
||||||
|
# 或使用 pnpm
|
||||||
|
pnpm install
|
||||||
|
```
|
||||||
|
|
||||||
|
### 开发模式
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 启动开发服务器（支持热重载）
|
||||||
|
wails3 dev
|
||||||
|
```
|
||||||
|
|
||||||
|
应用将在开发模式下启动，前后端修改会自动重载。
|
||||||
|
|
||||||
|
### 生产构建
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 构建生产版本
|
||||||
|
wails3 build
|
||||||
|
|
||||||
|
# 构建产物位于 build/ 目录
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📚 使用说明
|
||||||
|
|
||||||
|
### 基本使用流程
|
||||||
|
|
||||||
|
1. **启动应用**
|
||||||
|
```bash
|
||||||
|
wails3 dev # 或运行打包后的可执行文件
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **输入目标网站 URL**
|
||||||
|
- 在界面中输入要克隆的网站地址
|
||||||
|
- 例如：`https://example.com`
|
||||||
|
|
||||||
|
3. **获取资源列表**
|
||||||
|
- 点击「获取资源」按钮
|
||||||
|
- 系统自动分析站点并列出所有资源
|
||||||
|
|
||||||
|
4. **下载站点**
|
||||||
|
- 选择需要下载的资源类型
|
||||||
|
- 点击「下载」开始克隆
|
||||||
|
- 实时查看下载进度
|
||||||
|
|
||||||
|
5. **本地预览**
|
||||||
|
- 从下载列表中选择站点
|
||||||
|
- 点击「打开文件夹」查看文件
|
||||||
|
- 或使用集成的 Nginx 服务进行本地部署
|
||||||
|
|
||||||
|
### 配置文件说明
|
||||||
|
|
||||||
|
编辑 `config.yaml` 自定义配置：
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
# 应用名称
|
||||||
|
appName: "Go Site Clone"
|
||||||
|
|
||||||
|
# 本地服务端口
|
||||||
|
port: 6997
|
||||||
|
|
||||||
|
# WebSocket 连接地址
|
||||||
|
wsUrl: "106.12.33.188:6996"
|
||||||
|
|
||||||
|
# STUN 服务器地址
|
||||||
|
stunUrl: "stun:106.12.33.188:3478"
|
||||||
|
|
||||||
|
# API 服务地址
|
||||||
|
apiUrl: "http://106.12.33.188:6996"
|
||||||
|
|
||||||
|
# 网站文件保存目录
|
||||||
|
siteFileDir: "www"
|
||||||
|
```
|
||||||
|
|
||||||
|
### API 接口
|
||||||
|
|
||||||
|
应用暴露以下方法供前端调用：
|
||||||
|
|
||||||
|
#### `GetResources(url string)`
|
||||||
|
获取指定 URL 的所有资源列表
|
||||||
|
|
||||||
|
**参数:**
|
||||||
|
- `url`: 目标网站地址
|
||||||
|
|
||||||
|
**返回:** `ResourcesList` 对象，包含分类资源
|
||||||
|
|
||||||
|
#### `DownloadSite(uri string, resources ResourcesList)`
|
||||||
|
下载网站资源到本地
|
||||||
|
|
||||||
|
**参数:**
|
||||||
|
- `uri`: 网站地址
|
||||||
|
- `resources`: 资源列表对象
|
||||||
|
|
||||||
|
**返回:** `bool` - 下载是否成功
|
||||||
|
|
||||||
|
#### `GetDownloadList()`
|
||||||
|
获取已下载的网站列表
|
||||||
|
|
||||||
|
**返回:** `[]FileDir` - 文件目录列表
|
||||||
|
|
||||||
|
#### `OpenSiteFileDir(path string)`
|
||||||
|
打开指定网站的文件夹
|
||||||
|
|
||||||
|
**参数:**
|
||||||
|
- `path`: 相对路径
|
||||||
|
|
||||||
|
**返回:** `bool` - 操作是否成功
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🏗️ 技术架构
|
||||||
|
|
||||||
|
### 技术栈
|
||||||
|
|
||||||
|
**后端 (Go)**
|
||||||
|
- [Wails v3](https://wails.io/) - 应用框架
|
||||||
|
- [go-rod](https://github.com/go-rod/rod) - 浏览器自动化
|
||||||
|
- [go-rod/stealth](https://github.com/go-rod/stealth) - 反检测
|
||||||
|
- [goquery](https://github.com/PuerkitoBio/goquery) - HTML 解析
|
||||||
|
- [go-sqlite3](https://github.com/mattn/go-sqlite3) - 数据库
|
||||||
|
- [yaml.v3](https://github.com/go-yaml/yaml) - 配置管理
|
||||||
|
|
||||||
|
**前端 (Nuxt.js)**
|
||||||
|
- [Nuxt 4](https://nuxt.com/) - Vue 框架
|
||||||
|
- [Vue 3](https://vuejs.org/) - 响应式框架
|
||||||
|
- [Ant Design Vue](https://antdv.com/) - UI 组件库
|
||||||
|
- [Chart.js](https://www.chartjs.org/) - 图表库
|
||||||
|
- [Wails Runtime](https://wails.io/) - 前后端通信
|
||||||
|
|
||||||
|
### 项目结构
|
||||||
|
|
||||||
|
```
|
||||||
|
go-site-clone/
|
||||||
|
├── app.go # 应用主逻辑
|
||||||
|
├── main.go # 程序入口
|
||||||
|
├── config.yaml # 配置文件
|
||||||
|
├── go.mod # Go 模块依赖
|
||||||
|
│
|
||||||
|
├── config/ # 配置加载模块
|
||||||
|
│ └── index.go
|
||||||
|
│
|
||||||
|
├── services/ # 业务服务层
|
||||||
|
│ ├── site_service.go # 网站克隆核心服务
|
||||||
|
│ └── nginx_service.go # Nginx 服务管理
|
||||||
|
│
|
||||||
|
├── libs/ # 核心库
|
||||||
|
│ ├── browser.go # 浏览器池管理
|
||||||
|
│ ├── chrome.go # Chrome 实例创建
|
||||||
|
│ ├── sqlite.go # 数据库操作
|
||||||
|
│ ├── pack.go # 打包工具
|
||||||
|
│ └── browser-fingerprint/ # 浏览器指纹伪装
|
||||||
|
│ ├── index.go
|
||||||
|
│ ├── canvas.go # Canvas 指纹
|
||||||
|
│ └── timezone.go # 时区模拟
|
||||||
|
│
|
||||||
|
├── types/ # 类型定义
|
||||||
|
│ ├── fingerprint_type.go # 指纹参数类型
|
||||||
|
│ ├── task_type.go # 任务类型
|
||||||
|
│ └── nginx_template.go # Nginx 模板
|
||||||
|
│
|
||||||
|
├── utils/ # 工具函数
|
||||||
|
│ ├── file.go # 文件下载/管理
|
||||||
|
│ └── index.go # 环境检查
|
||||||
|
│
|
||||||
|
├── frontend/ # Nuxt.js 前端
|
||||||
|
│ ├── nuxt.config.ts # Nuxt 配置
|
||||||
|
│ ├── package.json # 前端依赖
|
||||||
|
│ ├── app/ # 应用代码
|
||||||
|
│ │ ├── app.vue # 根组件
|
||||||
|
│ │ ├── components/ # Vue 组件
|
||||||
|
│ │ ├── pages/ # 页面路由
|
||||||
|
│ │ ├── layouts/ # 布局模板
|
||||||
|
│ │ └── plugins/ # 插件
|
||||||
|
│ └── bindings/ # Go 绑定类型
|
||||||
|
│
|
||||||
|
├── user-data/ # 浏览器用户数据
|
||||||
|
├── www/ # 网站文件存储目录
|
||||||
|
├── plugin/ # 插件
|
||||||
|
│ └── nginx/ # Nginx 插件
|
||||||
|
└── build/ # 构建配置
|
||||||
|
├── darwin/ # macOS 打包配置
|
||||||
|
├── linux/ # Linux 打包配置
|
||||||
|
└── windows/ # Windows 打包配置
|
||||||
|
```
|
||||||
|
|
||||||
|
### 核心模块说明
|
||||||
|
|
||||||
|
#### 1. 浏览器管理 (`libs/browser.go`)
|
||||||
|
- 浏览器实例池化
|
||||||
|
- 并发安全的读写锁
|
||||||
|
- 自动创建和销毁
|
||||||
|
|
||||||
|
#### 2. 网站克隆服务 (`services/site_service.go`)
|
||||||
|
- 资源抓取引擎
|
||||||
|
- 智能链接遍历
|
||||||
|
- 资源去重和过滤
|
||||||
|
|
||||||
|
#### 3. 指纹伪装 (`libs/browser-fingerprint/`)
|
||||||
|
- Canvas 指纹随机化
|
||||||
|
- 时区和语言模拟
|
||||||
|
- UserAgent 定制
|
||||||
|
|
||||||
|
#### 4. 文件管理 (`utils/file.go`)
|
||||||
|
- 文件下载和存储
|
||||||
|
- 目录结构管理
|
||||||
|
- 文件列表查询
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ⚙️ 配置说明
|
||||||
|
|
||||||
|
### 浏览器指纹配置
|
||||||
|
|
||||||
|
```go
|
||||||
|
type BrowserFingerprintParams struct {
|
||||||
|
Canvas bool // 是否随机化 Canvas 指纹
|
||||||
|
TimeZone string // 时区，如 "America/New_York"
|
||||||
|
Language string // 语言，如 "en-US"
|
||||||
|
UserAgent string // 自定义 UserAgent
|
||||||
|
GeoLocation string // 地理位置，如 "40.7128,-74.0060"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### 任务参数配置
|
||||||
|
|
||||||
|
```go
|
||||||
|
type TaskParams struct {
|
||||||
|
URL string // 目标网站 URL
|
||||||
|
Fingerprint BrowserFingerprintParams // 指纹参数
|
||||||
|
Proxy string // 代理地址（可选）
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔧 开发指南
|
||||||
|
|
||||||
|
### 添加新功能
|
||||||
|
|
||||||
|
1. **后端服务**
|
||||||
|
- 在 `services/` 目录创建新服务
|
||||||
|
- 实现业务逻辑
|
||||||
|
- 在 `app.go` 中暴露方法给前端
|
||||||
|
|
||||||
|
2. **前端界面**
|
||||||
|
- 在 `frontend/app/pages/` 添加新页面
|
||||||
|
- 在 `frontend/app/components/` 添加组件
|
||||||
|
- 使用 Wails Runtime 调用后端方法
|
||||||
|
|
||||||
|
### 调试技巧
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 查看详细日志
|
||||||
|
wails3 dev -v
|
||||||
|
|
||||||
|
# 构建时查看详细信息
|
||||||
|
wails3 build -v
|
||||||
|
|
||||||
|
# 清理构建缓存
|
||||||
|
wails3 build -clean
|
||||||
|
```
|
||||||
|
|
||||||
|
### 测试
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 运行 Go 测试
|
||||||
|
go test ./...
|
||||||
|
|
||||||
|
# 运行前端测试（如果配置）
|
||||||
|
cd frontend
|
||||||
|
npm test
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📦 打包部署
|
||||||
|
|
||||||
|
### Windows
|
||||||
|
|
||||||
|
```bash
|
||||||
|
wails3 build -platform windows/amd64
|
||||||
|
```
|
||||||
|
|
||||||
|
生成的可执行文件位于 `build/bin/`
|
||||||
|
|
||||||
|
### macOS
|
||||||
|
|
||||||
|
```bash
|
||||||
|
wails3 build -platform darwin/universal
|
||||||
|
```
|
||||||
|
|
||||||
|
生成 `.app` 应用包
|
||||||
|
|
||||||
|
### Linux
|
||||||
|
|
||||||
|
```bash
|
||||||
|
wails3 build -platform linux/amd64
|
||||||
|
```
|
||||||
|
|
||||||
|
支持生成 `.deb`、`.rpm`、AppImage 等格式
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🤝 贡献指南
|
||||||
|
|
||||||
|
欢迎提交 Issue 和 Pull Request！
|
||||||
|
|
||||||
|
1. Fork 本仓库
|
||||||
|
2. 创建特性分支 (`git checkout -b feature/AmazingFeature`)
|
||||||
|
3. 提交更改 (`git commit -m 'Add some AmazingFeature'`)
|
||||||
|
4. 推送到分支 (`git push origin feature/AmazingFeature`)
|
||||||
|
5. 提交 Pull Request
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📄 许可证
|
||||||
|
|
||||||
|
本项目采用 MIT 许可证 - 详见 [LICENSE](LICENSE) 文件
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🙏 致谢
|
||||||
|
|
||||||
|
- [Wails](https://wails.io/) - 出色的 Go + Web 框架
|
||||||
|
- [go-rod](https://go-rod.github.io/) - 强大的浏览器自动化工具
|
||||||
|
- [Nuxt.js](https://nuxt.com/) - 优雅的 Vue 框架
|
||||||
|
- [Ant Design Vue](https://antdv.com/) - 精美的 UI 组件库
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📞 联系方式
|
||||||
|
|
||||||
|
- 作者: zhuy1228
|
||||||
|
- 项目链接: [https://github.com/zhuy1228/go-site-clone](https://github.com/zhuy1228/go-site-clone)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
<div align="center">
|
||||||
|
|
||||||
|
**如果这个项目对你有帮助，请给它一个 ⭐️**
|
||||||
|
|
||||||
|
Made with ❤️ by zhuy1228
|
||||||
|
|
||||||
|
</div>
|
||||||
Reference in New Issue
Block a user