@@ -102,7 +96,7 @@
- ✅ Customize the template to generate the channel you want
- ✅ Supports multiple source acquisition methods: multicast source, hotel source, subscription source, keyword search
-- ✅ Interface speed testing and verification, with priority on response time and resolution, filtering out ineffective interfaces
+- ✅ Interface speed verification, obtain delay, speed, resolution, filter invalid interface
- ✅ Preferences: IPv6, priority and quantity of interface source sorting, and interface whitelist
- ✅ Scheduled execution at 6:00 AM and 18:00 PM Beijing time daily
- ✅ Supports various execution methods: workflows, command line, GUI software, Docker(amd64/arm64/arm v7)
@@ -134,37 +128,39 @@ https://ghp.ci/raw.githubusercontent.com/Guovin/iptv-api/gd/source.json
### Method 1: Workflow
-Fork this project and initiate workflow updates, detailed steps are available at [Detailed Tutorial](./docs/tutorial_en.md)
+Fork this project and initiate workflow updates, detailed steps are available
+at [Detailed Tutorial](./docs/tutorial_en.md)
### Method 2: Command Line
-```python
+```shell
pip install pipenv
```
-```python
+```shell
pipenv install --dev
```
Start update:
-```python
+```shell
pipenv run dev
```
Start service:
-```python
+```shell
pipenv run service
```
### Method 3: GUI Software
-1. Download [IPTV-API update software](https://github.com/Guovin/iptv-api/releases), open the software, click update to complete the update
+1. Download [IPTV-API update software](https://github.com/Guovin/iptv-api/releases), open the software, click update to
+ complete the update
2. Or run the following command in the project directory to open the GUI software:
-```python
+```shell
pipenv run ui
```
@@ -172,8 +168,11 @@ pipenv run ui
### Method 4: Docker
-- iptv-api (Full version): Higher performance requirements, slower update speed, high stability and success rate. Set open_driver = False to switch to the lite running mode (recommended for hotel sources, multicast sources, and online searches)
-- iptv-api:lite (Condensed version): Lightweight, low performance requirements, fast update speed, stability uncertain (recommend using this version for the subscription source)
+- iptv-api (Full version): Higher performance requirements, slower update speed, high stability and success rate. Set
+ open_driver = False to switch to the lite running mode (recommended for hotel sources, multicast sources, and online
+ searches)
+- iptv-api:lite (Condensed version): Lightweight, low performance requirements, fast update speed, stability uncertain (
+ recommend using this version for the subscription source)
It's recommended to try each one and choose the version that suits you
@@ -206,7 +205,8 @@ docker run -d -p 8000:8000 guovern/iptv-api:lite
```
Volume Mount Parameter (Optional):
-This allows synchronization of files between the host machine and the container. Modifying templates, configurations, and retrieving updated result files can be directly operated in the host machine's folder.
+This allows synchronization of files between the host machine and the container. Modifying templates, configurations,
+and retrieving updated result files can be directly operated in the host machine's folder.
Taking the host path /etc/docker as an example:
@@ -239,7 +239,7 @@ docker run -v /etc/docker/config:/iptv-api-lite/config -v /etc/docker/output:/ip
Development and maintenance are not easy, please buy me a coffee ~
| Alipay | Wechat |
-| ------------------------------------- | ----------------------------------------- |
+|---------------------------------------|-------------------------------------------|
|  |  |
## Follow
@@ -250,7 +250,8 @@ Wechat public account search for Govin, or scan the code to receive updates and
## Disclaimer
-This project is for learning and communication purposes only. All interface data comes from the internet. If there is any infringement, please contact us for removal.
+This project is for learning and communication purposes only. All interface data comes from the internet. If there is
+any infringement, please contact us for removal.
## License
diff --git a/config/whitelist.txt b/config/whitelist.txt
new file mode 100644
index 0000000000..d11d6c4495
--- /dev/null
+++ b/config/whitelist.txt
@@ -0,0 +1,4 @@
+# 这是接口或订阅源的白名单，白名单内的接口或订阅源获取的接口将不会参与测速，优先排序至结果最前。
+# 填写频道名称会直接保留该记录至最终结果，如：CCTV-1,接口地址，只填写接口地址则对所有频道生效，多条记录换行输入。
+# This is the whitelist of the interface or subscription source. The interface in the whitelist or the interface obtained by the subscription source will not participate in the speed measurement and will be prioritized in the result.
+# Filling in the channel name will directly retain the record to the final result, such as: CCTV-1, interface address, only fill in the interface address will be effective for all channels, multiple records newline input.
diff --git a/docs/tutorial.md b/docs/tutorial.md
index 1ec93e5221..1ee117287d 100644
--- a/docs/tutorial.md
+++ b/docs/tutorial.md
@@ -58,7 +58,9 @@
1. 创建文件
2. 模板文件命名为 user_demo.txt
-3. 模板文件需要按照（频道分类,#genre#），（频道名称,频道接口）进行编写，注意是英文逗号。如果需要将该接口设为白名单（不测速、保留在结果最前），可在地址后添加$!即可，例如http://xxx$!。后面也可以添加额外说明信息，如：http://xxx$!白名单接口
+3.
+模板文件需要按照（频道分类,#genre#），（频道名称,频道接口）进行编写，注意是英文逗号。如果需要将该接口设为白名单（不测速、保留在结果最前），可在地址后添加$!即可，例如http://xxx$!
+。后面也可以添加额外说明信息，如：http://xxx$!白名单接口
4. 点击 Commit changes...进行保存
@@ -81,8 +83,8 @@
2. 配置文件命名为 user_config.ini
3. 粘贴默认配置
4. 修改模板和结果文件配置：
- - source_file = config/user_demo.txt
- - final_file = output/user_result.txt
+ - source_file = config/user_demo.txt
+ - final_file = output/user_result.txt
5. 点击 Commit changes...进行保存
@@ -124,7 +126,8 @@
这个时候就可以运行更新工作流了
1. 点击 Run workflow
-2. 这里可以切换您要运行的仓库分支，由于 Fork 默认拉取的是 master 分支，如果您修改的模板和配置也在 master 分支，这里选择 master 就好了，点击 Run workflow 确认运行
+2. 这里可以切换您要运行的仓库分支，由于 Fork 默认拉取的是 master 分支，如果您修改的模板和配置也在 master 分支，这里选择
+ master 就好了，点击 Run workflow 确认运行
@@ -132,7 +135,8 @@
稍等片刻，就可以看到您的第一条更新工作流已经在运行了！
-（注意：由于运行时间取决于您的模板频道数量以及页数等配置，也很大程度取决于当前网络状况，请耐心等待，默认模板与配置一般需要 25 分钟左右。）
+（注意：由于运行时间取决于您的模板频道数量以及页数等配置，也很大程度取决于当前网络状况，请耐心等待，默认模板与配置一般需要 25
+分钟左右。）
#### （4）Workflow 取消运行：
@@ -147,9 +151,11 @@
https://ghp.ci/raw.githubusercontent.com/您的github用户名/仓库名称（对应上述Fork创建时的TV）/master/output/user_result.txt
-如果访问该链接能正常返回更新后的接口内容，说明您的直播源接口链接已经大功告成了！将该链接复制粘贴到 TVBox 等软件配置栏中即可使用~
+如果访问该链接能正常返回更新后的接口内容，说明您的直播源接口链接已经大功告成了！将该链接复制粘贴到 TVBox
+等软件配置栏中即可使用~
-- 注意：除了首次执行工作流需要您手动触发，后续执行（默认北京时间每日 6:00 与 18:00）将自动触发。如果您修改了模板或配置文件想立刻执行更新，可手动触发（2）中的 Run workflow 即可。
+- 注意：除了首次执行工作流需要您手动触发，后续执行（默认北京时间每日 6:00 与 18:00）将自动触发。如果您修改了模板或配置文件想立刻执行更新，可手动触发（2）中的
+ Run workflow 即可。
### 4.修改工作流更新频率（可选）
@@ -174,23 +180,23 @@ https://ghp.ci/raw.githubusercontent.com/您的github用户名/仓库名称（
2. 运行更新
项目目录下打开终端 CMD 依次运行以下命令：
-```python
+```shell
pip install pipenv
```
-```python
+```shell
pipenv install --dev
```
启动更新：
-```python
+```shell
pipenv run dev
```
启动服务：
-```python
+```shell
pipenv run service
```
@@ -200,7 +206,7 @@ pipenv run service
2. 或者在项目目录下运行以下命令，即可打开 GUI 软件：
-```python
+```shell
pipenv run ui
```
@@ -208,7 +214,8 @@ pipenv run ui
### 方式四：Docker
-- iptv-api（完整版本）：性能要求较高，更新速度较慢，稳定性、成功率高；修改配置 open_driver = False 可切换到 Lite 版本运行模式（推荐酒店源、组播源、关键字搜索使用此版本）
+- iptv-api（完整版本）：性能要求较高，更新速度较慢，稳定性、成功率高；修改配置 open_driver = False 可切换到 Lite
+ 版本运行模式（推荐酒店源、组播源、关键字搜索使用此版本）
- iptv-api:lite（精简版本）：轻量级，性能要求低，更新速度快，稳定性不确定（推荐订阅源使用此版本）
1. 拉取镜像：
diff --git a/docs/tutorial_en.md b/docs/tutorial_en.md
index f347433d14..e31df28e90 100644
--- a/docs/tutorial_en.md
+++ b/docs/tutorial_en.md
@@ -12,18 +12,21 @@ Copy the source code of this repository to your personal account repository
### 2. Fork to create a personal repository:
-1. Name your personal repository, you can name it whatever you like (the final live source result link depends on this name), here we use default TV as an example.
+1. Name your personal repository, you can name it whatever you like (the final live source result link depends on this
+ name), here we use default TV as an example.
2. After confirming the information is correct, click to confirm and create.
## Step 2: Update the Source Code
-Since this project will continue to iterate and improve, if you want to get the latest updates, you can do the following:
+Since this project will continue to iterate and improve, if you want to get the latest updates, you can do the
+following:
### 1. Star
-Go to https://github.com/Guovin/iptv-api, click on Star to bookmark this project (Your Star is my motivation to keep updating).
+Go to https://github.com/Guovin/iptv-api, click on Star to bookmark this project (Your Star is my motivation to keep
+updating).
### 2. Watch
@@ -35,17 +38,20 @@ Follow this project to be notified by email about the latest updates and release
#### Normal update:
-Go back to the homepage of your repository after forking. If there are updates to the project, click on "Sync fork" and then "Update branch" to confirm and update to the latest code.
+Go back to the homepage of your repository after forking. If there are updates to the project, click on "Sync fork" and
+then "Update branch" to confirm and update to the latest code.
#### No Update branch button, update conflict:
-This is because some files conflict with the default files in the main repository. Click "Discard commits" to update to the latest code.
+This is because some files conflict with the default files in the main repository. Click "Discard commits" to update to
+the latest code.
## Step 3: Modify the Template
-When you click to confirm and create in step one, you will be automatically redirected to your personal repository. At this point, your personal repository is created, and you can customize your personal live source channel menu!
+When you click to confirm and create in step one, you will be automatically redirected to your personal repository. At
+this point, your personal repository is created, and you can customize your personal live source channel menu!
### 1. Click the demo.txt template file inside the config folder:
@@ -55,7 +61,10 @@ When you click to confirm and create in step one, you will be automatically redi
1. Create file
2. Name the template file user_demo.txt
-3. The template file needs to be written in the format of (channel category, #genre#), (channel name, channel interface), note that it is an English comma. If you need to set this interface as a whitelist (no speed testing, kept at the front of the results), simply add $! after the address. For example: http://xxx$!. Additional information can also be appended afterward, such as: http://xxx$! Whitelist interface.
+3. The template file needs to be written in the format of (channel category, #genre#), (channel name, channel
+ interface), note that it is an English comma. If you need to set this interface as a whitelist (no speed testing,
+ kept at the front of the results), simply add $! after the address. For example: http://xxx$!. Additional information
+ can also be appended afterward, such as: http://xxx$! Whitelist interface.
4. Click Commit changes... to save
@@ -78,8 +87,8 @@ Similar to editing the template, modify the running configuration
2. Name the configuration file user_config.ini
3. Paste the default template
4. Modify the template and result file configuration:
- - source_file = config/user_demo.txt
- - final_file = output/user_result.txt
+ - source_file = config/user_demo.txt
+ - final_file = output/user_result.txt
5. Click Commit changes... to save
@@ -102,24 +111,29 @@ If your template and configuration modifications are correct, you can configure
### 2. Enable Actions workflow:
-Since the Actions workflow of the forked repository is disabled by default, you need to manually enable it by clicking the button in the red box to confirm.
+Since the Actions workflow of the forked repository is disabled by default, you need to manually enable it by clicking
+the button in the red box to confirm.
-After successful activation, you can see that there are no workflows running at the moment. Don't worry, let's start running your first update workflow below.
+After successful activation, you can see that there are no workflows running at the moment. Don't worry, let's start
+running your first update workflow below.
### 3. Run the update workflow:
#### (1) Enable update schedule:
1. Click on update schedule under the Workflows category.
-2. Since the workflow of the forked repository is disabled by default, click the Enable workflow button to confirm the activation.
+2. Since the workflow of the forked repository is disabled by default, click the Enable workflow button to confirm the
+ activation.
#### (2) Run the Workflow based on branches:
1. Click Run workflow.
-2. Here you can switch to the branch you want to run. Since the fork defaults to the master branch, if the template and configuration you modified are also in the master branch, just choose master here, and click Run workflow to confirm the run.
+2. Here you can switch to the branch you want to run. Since the fork defaults to the master branch, if the template and
+ configuration you modified are also in the master branch, just choose master here, and click Run workflow to confirm
+ the run.
Now you can run the update workflow.
@@ -127,30 +141,40 @@ Now you can run the update workflow.
#### (3) Workflow in progress:
Wait a moment, and you will see that your first update workflow is running!
-(Note: The running time depends on the number of channels and pages in your template and other configurations, and also largely depends on the current network conditions. Please be patient. The default template and configuration usually take about 25 minutes.)
+(Note: The running time depends on the number of channels and pages in your template and other configurations, and also
+largely depends on the current network conditions. Please be patient. The default template and configuration usually
+take about 25 minutes.)
#### (4) Cancel the running Workflow:
-If you feel that this update is not quite right and you need to modify the template or configuration before running again, you can click Cancel run to cancel this run.
+If you feel that this update is not quite right and you need to modify the template or configuration before running
+again, you can click Cancel run to cancel this run.
#### (5) Workflow executed successfully:
-If everything is normal, after a short wait, you will see that the workflow has been executed successfully (green check mark).
+If everything is normal, after a short wait, you will see that the workflow has been executed successfully (green check
+mark).
At this point, you can visit the proxy file link to see if the latest results have been synchronized:
-https://ghp.ci/raw.githubusercontent.com/your github username/repository name (corresponding to the TV created when forking)/master/user_result.txt
+https://ghp.ci/raw.githubusercontent.com/your github username/repository name (corresponding to the TV created when
+forking)/master/user_result.txt
-If you can access this link and it returns the updated interface content, then your live source interface link has been successfully created! Simply copy and paste this link into software like TVBox in the configuration field to use~
+If you can access this link and it returns the updated interface content, then your live source interface link has been
+successfully created! Simply copy and paste this link into software like TVBox in the configuration field to use~
-- Note: Except for the first execution of the workflow, which requires you to manually trigger it, subsequent executions (default: 6:00 AM and 18:00 PM Beijing time daily) will be automatically triggered. If you have modified the template or configuration files and want to execute the update immediately, you can manually trigger (2) Run workflow.
+- Note: Except for the first execution of the workflow, which requires you to manually trigger it, subsequent
+ executions (default: 6:00 AM and 18:00 PM Beijing time daily) will be automatically triggered. If you have modified
+ the template or configuration files and want to execute the update immediately, you can manually trigger (2) Run
+ workflow.
### 4.Modify Workflow Update Frequency(optional)
-If you want to modify the update frequency (default: 6:00 AM and 18:00 PM Beijing time daily), you can modify the on:schedule:- cron field.
+If you want to modify the update frequency (default: 6:00 AM and 18:00 PM Beijing time daily), you can modify the on:
+schedule:- cron field.
If you want to perform updates every 2 days, you can modify it like this:
@@ -166,38 +190,40 @@ If you want to perform updates every 2 days, you can modify it like this:
### Method 2: Command Line
1. Install Python
- Please download and install Python from the official site. During installation, choose to add Python to the system's environment variables Path.
+ Please download and install Python from the official site. During installation, choose to add Python to the system's
+ environment variables Path.
2. Run Update
Open a CMD terminal in the project directory and run the following commands in sequence:
-```python
+```shell
pip install pipenv
```
-```python
+```shell
pipenv install --dev
```
Start update:
-```python
+```shell
pipenv run dev
```
Start service:
-```python
+```shell
pipenv run service
```
### Method 3: GUI Software
-1. Download [IPTV-API software](https://github.com/Guovin/iptv-api/releases), open the software, click update to complete the update.
+1. Download [IPTV-API software](https://github.com/Guovin/iptv-api/releases), open the software, click update to
+ complete the update.
2. Alternatively, run the following command in the project directory to open the GUI software:
-```python
+```shell
pipenv run ui
```
@@ -205,8 +231,11 @@ pipenv run ui
### Method 4: Docker
-- iptv-api (Full version): Higher performance requirements, slower update speed, high stability and success rate. Set open_driver = False to switch to the lite running mode (recommended for hotel sources, multicast sources, and online searches)
-- iptv-api:lite (Condensed version): Lightweight, low performance requirements, fast update speed, stability uncertain (recommend using this version for the subscription source)
+- iptv-api (Full version): Higher performance requirements, slower update speed, high stability and success rate. Set
+ open_driver = False to switch to the lite running mode (recommended for hotel sources, multicast sources, and online
+ searches)
+- iptv-api:lite (Condensed version): Lightweight, low performance requirements, fast update speed, stability uncertain (
+ recommend using this version for the subscription source)
It's recommended to try each one and choose the version that suits you
@@ -239,7 +268,8 @@ docker run -d -p 8000:8000 guovern/iptv-api:lite
```
Volume Mount Parameter (Optional):
-This allows synchronization of files between the host machine and the container. Modifying templates, configurations, and retrieving updated result files can be directly operated in the host machine's folder.
+This allows synchronization of files between the host machine and the container. Modifying templates, configurations,
+and retrieving updated result files can be directly operated in the host machine's folder.
Taking the host path /etc/docker as an example:
@@ -265,6 +295,8 @@ docker run -v /etc/docker/config:/iptv-api-lite/config -v /etc/docker/output:/ip
### Update the File to the Repository(optional)
-If you do not have your own domain address, after the interface update is completed, upload user_result.txt to your personal repository to use it.
-https://ghp.ci/raw.githubusercontent.com/your github username/repository name (corresponding to the TV created when forking)/master/output/user_result.txt
+If you do not have your own domain address, after the interface update is completed, upload user_result.txt to your
+personal repository to use it.
+https://ghp.ci/raw.githubusercontent.com/your github username/repository name (corresponding to the TV created when
+forking)/master/output/user_result.txt
diff --git a/main.py b/main.py
index 7e2c276dec..d3ba78fcac 100644
--- a/main.py
+++ b/main.py
@@ -30,6 +30,7 @@
format_interval,
check_ipv6_support,
resource_path,
+ get_whitelist_urls
)
@@ -71,8 +72,9 @@ async def visit_page(self, channel_names=None):
if config.open_method[setting]:
if setting == "subscribe":
subscribe_urls = config.subscribe_urls
+ whitelist_urls = get_whitelist_urls()
task = asyncio.create_task(
- task_func(subscribe_urls, callback=self.update_progress)
+ task_func(subscribe_urls, whitelist=whitelist_urls, callback=self.update_progress)
)
elif setting == "hotel_foodie" or setting == "hotel_fofa":
task = asyncio.create_task(task_func(callback=self.update_progress))
diff --git a/tkinter_ui/tkinter_ui.spec b/tkinter_ui/tkinter_ui.spec
index 299ee87fda..29d406c943 100644
--- a/tkinter_ui/tkinter_ui.spec
+++ b/tkinter_ui/tkinter_ui.spec
@@ -7,6 +7,7 @@ a = Analysis(
datas=[
('../config/config.ini', 'config'),
('../config/demo.txt', 'config'),
+ ('../config/whitelist.txt', 'config'),
('../config/rtp', 'config/rtp'),
('../updates/hotel/cache.pkl', 'updates/hotel'),
('../updates/multicast/multicast_map.json', 'updates/multicast'),
diff --git a/updates/multicast/update_tmp.py b/updates/multicast/update_tmp.py
index 7eedc33055..fb88f07303 100644
--- a/updates/multicast/update_tmp.py
+++ b/updates/multicast/update_tmp.py
@@ -1,5 +1,5 @@
-import sys
import os
+import sys
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "../..")))
@@ -7,14 +7,13 @@
from driver.utils import get_soup_driver
from utils.config import config
import utils.constants as constants
-from utils.channel import format_channel_name, get_name_url
-from utils.tools import get_pbar_remaining, resource_path
+from utils.channel import format_channel_name
+from utils.tools import get_pbar_remaining, resource_path, get_name_url
import json
# import asyncio
from requests import Session
from collections import defaultdict
-import re
from time import time
from tqdm import tqdm
@@ -40,7 +39,7 @@ def get_region_urls_from_IPTV_Multicast_source():
region_url[name]["移动"] = mobile
region_url[name]["电信"] = telecom
with open(
- resource_path("updates/multicast/multicast_map.json"), "w", encoding="utf-8"
+ resource_path("updates/multicast/multicast_map.json"), "w", encoding="utf-8"
) as f:
json.dump(region_url, f, ensure_ascii=False, indent=4)
@@ -51,7 +50,7 @@ def get_multicast_urls_info_from_region_list():
"""
urls_info = []
with open(
- resource_path("updates/multicast/multicast_map.json"), "r", encoding="utf-8"
+ resource_path("updates/multicast/multicast_map.json"), "r", encoding="utf-8"
) as f:
region_url = json.load(f)
urls_info = [
@@ -71,9 +70,9 @@ async def get_multicast_region_result():
multicast_region_urls_info, multicast=True
)
with open(
- resource_path("updates/multicast/multicast_region_result.json"),
- "w",
- encoding="utf-8",
+ resource_path("updates/multicast/multicast_region_result.json"),
+ "w",
+ encoding="utf-8",
) as f:
json.dump(multicast_result, f, ensure_ascii=False, indent=4)
@@ -83,7 +82,7 @@ def get_multicast_region_type_result_txt():
Get multicast region type result txt
"""
with open(
- resource_path("updates/multicast/multicast_map.json"), "r", encoding="utf-8"
+ resource_path("updates/multicast/multicast_map.json"), "r", encoding="utf-8"
) as f:
region_url = json.load(f)
session = Session()
@@ -92,9 +91,9 @@ def get_multicast_region_type_result_txt():
response = session.get(url)
content = response.text
with open(
- resource_path(f"config/rtp/{region}_{type}.txt"),
- "w",
- encoding="utf-8",
+ resource_path(f"config/rtp/{region}_{type}.txt"),
+ "w",
+ encoding="utf-8",
) as f:
f.write(content)
@@ -109,11 +108,11 @@ def get_multicast_region_result_by_rtp_txt(callback=None):
filename.rsplit(".", 1)[0]
for filename in os.listdir(rtp_path)
if filename.endswith(".txt")
- and "_" in filename
- and (
- filename.rsplit(".", 1)[0].partition("_")[0] in config_region_list
- or config_region_list & {"all", "ALL", "全部"}
- )
+ and "_" in filename
+ and (
+ filename.rsplit(".", 1)[0].partition("_")[0] in config_region_list
+ or config_region_list & {"all", "ALL", "全部"}
+ )
]
total_files = len(rtp_file_list)
@@ -127,7 +126,7 @@ def get_multicast_region_result_by_rtp_txt(callback=None):
for filename in rtp_file_list:
region, _, type = filename.partition("_")
with open(
- os.path.join(rtp_path, f"{filename}.txt"), "r", encoding="utf-8"
+ os.path.join(rtp_path, f"{filename}.txt"), "r", encoding="utf-8"
) as f:
for line in f:
name_url = get_name_url(line, pattern=constants.rtp_pattern)
@@ -146,9 +145,9 @@ def get_multicast_region_result_by_rtp_txt(callback=None):
)
with open(
- resource_path("updates/multicast/multicast_region_result.json"),
- "w",
- encoding="utf-8",
+ resource_path("updates/multicast/multicast_region_result.json"),
+ "w",
+ encoding="utf-8",
) as f:
json.dump(multicast_result, f, ensure_ascii=False, indent=4)
diff --git a/updates/subscribe/request.py b/updates/subscribe/request.py
index 3133225771..9c12a3dfc2 100644
--- a/updates/subscribe/request.py
+++ b/updates/subscribe/request.py
@@ -1,28 +1,31 @@
-import utils.constants as constants
-from tqdm.asyncio import tqdm_asyncio
+from collections import defaultdict
+from concurrent.futures import ThreadPoolExecutor
from time import time
+
from requests import Session, exceptions
-from utils.config import config
+from tqdm.asyncio import tqdm_asyncio
+
import utils.constants as constants
+from utils.channel import format_channel_name
+from utils.config import config
from utils.retry import retry_func
-from utils.channel import get_name_url, format_channel_name
from utils.tools import (
merge_objects,
get_pbar_remaining,
format_url_with_cache,
add_url_info,
+ get_name_url
)
-from concurrent.futures import ThreadPoolExecutor
-from collections import defaultdict
async def get_channels_by_subscribe_urls(
- urls,
- multicast=False,
- hotel=False,
- retry=True,
- error_print=True,
- callback=None,
+ urls,
+ multicast=False,
+ hotel=False,
+ retry=True,
+ error_print=True,
+ whitelist=None,
+ callback=None,
):
"""
Get the channels by subscribe urls
@@ -53,6 +56,7 @@ def process_subscribe_channels(subscribe_info):
else:
subscribe_url = subscribe_info
channels = defaultdict(lambda: defaultdict(lambda: defaultdict(list)))
+ in_whitelist = whitelist and (subscribe_url in whitelist)
try:
response = None
try:
@@ -95,6 +99,8 @@ def process_subscribe_channels(subscribe_info):
else f"{subscribe_name}"
)
)
+ if in_whitelist:
+ info = "!" + info
url = add_url_info(url, info)
url = format_url_with_cache(
url, cache=subscribe_url if (multicast or hotel) else None
diff --git a/utils/channel.py b/utils/channel.py
index 805dfc92f6..9fe3602beb 100644
--- a/utils/channel.py
+++ b/utils/channel.py
@@ -17,6 +17,7 @@
sort_urls,
)
from utils.tools import (
+ get_name_url,
check_url_by_patterns,
get_total_urls,
process_nested_dict,
@@ -24,24 +25,12 @@
remove_cache_info,
resource_path,
write_content_into_txt,
+ get_whitelist_urls,
+ get_whitelist_name_urls
)
-def get_name_url(content, pattern, multiline=False, check_url=True):
- """
- Get channel name and url from content
- """
- flag = re.MULTILINE if multiline else 0
- matches = re.findall(pattern, content, flag)
- channels = [
- {"name": match[0].strip(), "url": match[1].strip()}
- for match in matches
- if (check_url and match[1].strip()) or not check_url
- ]
- return channels
-
-
-def get_channel_data_from_file(channels, file, use_old):
+def get_channel_data_from_file(channels, file, use_old, whitelist):
"""
Get the channel data from the file
"""
@@ -61,6 +50,9 @@ def get_channel_data_from_file(channels, file, use_old):
category_dict = channels[current_category]
if name not in category_dict:
category_dict[name] = []
+ if name in whitelist:
+ for whitelist_url in whitelist[name]:
+ category_dict[name].append((whitelist_url, None, None, "important"))
if use_old and url:
info = url.partition("$")[2]
origin = None
@@ -78,11 +70,15 @@ def get_channel_items():
"""
user_source_file = resource_path(config.source_file)
channels = defaultdict(lambda: defaultdict(list))
+ whitelist = get_whitelist_name_urls()
+ whitelist_len = len(list(whitelist.keys()))
+ if whitelist_len:
+ print(f"Found {whitelist_len} channel in whitelist")
if os.path.exists(user_source_file):
with open(user_source_file, "r", encoding="utf-8") as file:
channels = get_channel_data_from_file(
- channels, file, config.open_use_old_result
+ channels, file, config.open_use_old_result, whitelist
)
if config.open_use_old_result:
@@ -551,7 +547,10 @@ async def process_sort_channel_list(data, ipv6=False, callback=None):
"""
ipv6_proxy = None if (not config.open_ipv6 or ipv6) else constants.ipv6_proxy
need_sort_data = copy.deepcopy(data)
- process_nested_dict(need_sort_data, seen=set(), flag=r"cache:(.*)", force_str="!")
+ whitelist_urls = get_whitelist_urls()
+ if whitelist_urls:
+ print(f"Found {len(whitelist_urls)} whitelist urls")
+ process_nested_dict(need_sort_data, seen=set(whitelist_urls), flag=r"cache:(.*)", force_str="!")
result = {}
semaphore = asyncio.Semaphore(10)
@@ -574,7 +573,7 @@ async def limited_get_speed(info, ipv6_proxy, callback):
await asyncio.gather(*tasks)
for cate, obj in data.items():
for name, info_list in obj.items():
- info_list = sort_urls(name, info_list)
+ info_list = sort_urls(name, info_list, whitelist=whitelist_urls)
append_data_to_info_data(
result,
cate,
diff --git a/utils/config.py b/utils/config.py
index b49924169e..416c42a794 100644
--- a/utils/config.py
+++ b/utils/config.py
@@ -37,6 +37,7 @@ def get_resolution_value(resolution_str):
class ConfigManager:
def __init__(self):
+ self.config = None
self.load()
def __getattr__(self, name, *args, **kwargs):
diff --git a/utils/constants.py b/utils/constants.py
index c2b41e28d4..2ca97d4477 100644
--- a/utils/constants.py
+++ b/utils/constants.py
@@ -4,6 +4,8 @@
output_path = "output"
+whitelist_path = os.path.join(config_path, "whitelist.txt")
+
result_path = os.path.join(output_path, "result_new.txt")
cache_path = os.path.join(output_path, "cache.pkl")
diff --git a/utils/speed.py b/utils/speed.py
index 4ee0ab3fdd..3d1d8192c1 100644
--- a/utils/speed.py
+++ b/utils/speed.py
@@ -204,7 +204,7 @@ async def get_speed(url, ipv6_proxy=None, callback=None):
callback()
-def sort_urls(name, data, logger=None):
+def sort_urls(name, data, logger=None, whitelist=None):
"""
Sort the urls with info
"""
@@ -212,6 +212,8 @@ def sort_urls(name, data, logger=None):
if logger is None:
logger = get_logger(constants.sort_log_path, level=INFO, init=True)
for url, date, resolution, origin in data:
+ if whitelist and url in whitelist:
+ origin = "important"
result = {
"url": remove_cache_info(url),
"date": date,
diff --git a/utils/tools.py b/utils/tools.py
index f51033703a..7e20da2c93 100644
--- a/utils/tools.py
+++ b/utils/tools.py
@@ -7,6 +7,7 @@
import socket
import sys
import urllib.parse
+from collections import defaultdict
from logging.handlers import RotatingFileHandler
from time import time
@@ -525,3 +526,52 @@ def write_content_into_txt(content, path=None, newline=True, callback=None):
if callback:
callback()
+
+
+def get_name_url(content, pattern, multiline=False, check_url=True):
+ """
+ Get name and url from content
+ """
+ flag = re.MULTILINE if multiline else 0
+ matches = re.findall(pattern, content, flag)
+ channels = [
+ {"name": match[0].strip(), "url": match[1].strip()}
+ for match in matches
+ if (check_url and match[1].strip()) or not check_url
+ ]
+ return channels
+
+
+def get_whitelist_urls():
+ """
+ Get the whitelist urls
+ """
+ whitelist_file = resource_path(constants.whitelist_path)
+ urls = []
+ url_pattern = constants.url_pattern
+ if os.path.exists(whitelist_file):
+ with open(whitelist_file, "r", encoding="utf-8") as f:
+ for line in f:
+ match = re.search(url_pattern, line)
+ if match:
+ urls.append(match.group().strip())
+ return urls
+
+
+def get_whitelist_name_urls():
+ """
+ Get the whitelist name urls
+ """
+ whitelist_file = resource_path(constants.whitelist_path)
+ name_urls = defaultdict(list)
+ txt_pattern = constants.txt_pattern
+ if os.path.exists(whitelist_file):
+ with open(whitelist_file, "r", encoding="utf-8") as f:
+ for line in f:
+ name_url = get_name_url(line, pattern=txt_pattern)
+ if name_url and name_url[0]:
+ name = name_url[0]["name"]
+ url = name_url[0]["url"]
+ if url not in name_urls[name]:
+ name_urls[name].append(url)
+ return name_urls
- 
- 
-
-
- 
+ 