- 添加AK和SK的检测

- 添加检测规则提交入口
- 添加.gitignore文件
- 添加gitee高速下载通道
- 优化txt结果集输出方式
- 去除一些不必要的输出
- 更新apktool至最新版本
- 更新baksmali至最新版本
- 修复单dex文件不解析的问题
- 修复Windows下长文件不解析的问题
- 修复目录中包含空格无法解析的问题
- 修复WEB页面或者目录扫描的问题
- 修复使用默认python执行命令跳出应用商店问题
- 修复iOS应用使用IPA无结果的问题
- 修复因JS文件导致在进行AK搜索时出现的雪崩问题
v1.0.8 V1.0.8_Releases
kelvinBen 3 years ago
parent e8c12166a0
commit 72cd09be93
  1. 116
      .gitignore
  2. 128
      CODE_OF_CONDUCT.md
  3. 39
      README.md
  4. 1
      __init__.py
  5. 2
      app.py
  6. 12
      config.py
  7. 1
      libs/__init__.py
  8. 25
      libs/core/__init__.py
  9. 4
      libs/core/download.py
  10. 1
      libs/core/net.py
  11. 13
      libs/core/parses.py
  12. 1
      libs/task/__init__.py
  13. 7
      libs/task/android_task.py
  14. 18
      libs/task/base_task.py
  15. 1
      libs/task/download_task.py
  16. 7
      libs/task/ios_task.py
  17. 9
      libs/task/net_task.py
  18. 4
      libs/task/web_task.py
  19. BIN
      tools/apktool.jar
  20. BIN
      tools/baksmali.jar
  21. 12
      update.md

116
.gitignore vendored

@ -0,0 +1,116 @@
result_*.txt
result_*.xls
download/
history/
out/
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class
*.pyc
# C extensions
*.so
# Distribution / packaging
.Python
.DS_Store
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
.hypothesis/
.pytest_cache/
# Translations
*.mo
*.pot
# Django stuff:
*.log
local_settings.py
db.sqlite3
# Flask stuff:
instance/
.webassets-cache
# Scrapy stuff:
.scrapy
# Sphinx documentation
docs/_build/
# PyBuilder
target/
# Jupyter Notebook
.ipynb_checkpoints
# pyenv
.python-version
# celery beat schedule file
celerybeat-schedule
# SageMath parsed files
*.sage.py
# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/
# Spyder project settings
.spyderproject
.spyproject
# Rope project settings
.ropeproject
# mkdocs documentation
/site
# mypy
.mypy_cache/
# add
.idea/

@ -0,0 +1,128 @@
# Contributor Covenant Code of Conduct
## Our Pledge
We as members, contributors, and leaders pledge to make participation in our
community a harassment-free experience for everyone, regardless of age, body
size, visible or invisible disability, ethnicity, sex characteristics, gender
identity and expression, level of experience, education, socio-economic status,
nationality, personal appearance, race, religion, or sexual identity
and orientation.
We pledge to act and interact in ways that contribute to an open, welcoming,
diverse, inclusive, and healthy community.
## Our Standards
Examples of behavior that contributes to a positive environment for our
community include:
* Demonstrating empathy and kindness toward other people
* Being respectful of differing opinions, viewpoints, and experiences
* Giving and gracefully accepting constructive feedback
* Accepting responsibility and apologizing to those affected by our mistakes,
and learning from the experience
* Focusing on what is best not just for us as individuals, but for the
overall community
Examples of unacceptable behavior include:
* The use of sexualized language or imagery, and sexual attention or
advances of any kind
* Trolling, insulting or derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or email
address, without their explicit permission
* Other conduct which could reasonably be considered inappropriate in a
professional setting
## Enforcement Responsibilities
Community leaders are responsible for clarifying and enforcing our standards of
acceptable behavior and will take appropriate and fair corrective action in
response to any behavior that they deem inappropriate, threatening, offensive,
or harmful.
Community leaders have the right and responsibility to remove, edit, or reject
comments, commits, code, wiki edits, issues, and other contributions that are
not aligned to this Code of Conduct, and will communicate reasons for moderation
decisions when appropriate.
## Scope
This Code of Conduct applies within all community spaces, and also applies when
an individual is officially representing the community in public spaces.
Examples of representing our community include using an official e-mail address,
posting via an official social media account, or acting as an appointed
representative at an online or offline event.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported to the community leaders responsible for enforcement at
blsm@vip.qq.com.
All complaints will be reviewed and investigated promptly and fairly.
All community leaders are obligated to respect the privacy and security of the
reporter of any incident.
## Enforcement Guidelines
Community leaders will follow these Community Impact Guidelines in determining
the consequences for any action they deem in violation of this Code of Conduct:
### 1. Correction
**Community Impact**: Use of inappropriate language or other behavior deemed
unprofessional or unwelcome in the community.
**Consequence**: A private, written warning from community leaders, providing
clarity around the nature of the violation and an explanation of why the
behavior was inappropriate. A public apology may be requested.
### 2. Warning
**Community Impact**: A violation through a single incident or series
of actions.
**Consequence**: A warning with consequences for continued behavior. No
interaction with the people involved, including unsolicited interaction with
those enforcing the Code of Conduct, for a specified period of time. This
includes avoiding interactions in community spaces as well as external channels
like social media. Violating these terms may lead to a temporary or
permanent ban.
### 3. Temporary Ban
**Community Impact**: A serious violation of community standards, including
sustained inappropriate behavior.
**Consequence**: A temporary ban from any sort of interaction or public
communication with the community for a specified period of time. No public or
private interaction with the people involved, including unsolicited interaction
with those enforcing the Code of Conduct, is allowed during this period.
Violating these terms may lead to a permanent ban.
### 4. Permanent Ban
**Community Impact**: Demonstrating a pattern of violation of community
standards, including sustained inappropriate behavior, harassment of an
individual, or aggression toward or disparagement of classes of individuals.
**Consequence**: A permanent ban from any sort of public interaction within
the community.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant][homepage],
version 2.0, available at
https://www.contributor-covenant.org/version/2/0/code_of_conduct.html.
Community Impact Guidelines were inspired by [Mozilla's code of conduct
enforcement ladder](https://github.com/mozilla/diversity).
[homepage]: https://www.contributor-covenant.org
For answers to common questions about this code of conduct, see the FAQ at
https://www.contributor-covenant.org/faq. Translations are available at
https://www.contributor-covenant.org/translations.

@ -1,4 +1,4 @@
![Language](https://img.shields.io/badge/Language-Python3-blue) ![License](https://img.shields.io/badge/License-GPL3.0-orange)
![License](https://img.shields.io/badge/Version-V1.0.8-red) ![Language](https://img.shields.io/badge/Language-Python3-blue) ![License](https://img.shields.io/badge/License-GPL3.0-orange)
该项目目前仅仅是规划项目中的冰山一角,如果您对此项目感兴趣或者想参与后继项目的开发工作或者翻译工作中,请发送邮件至blsm@vip.qq.com说明你的能力和诉求。
@ -90,6 +90,11 @@ AppInfoScanner
或者复制以下链接到浏览器下载最新正式版本
https://github.com/kelvinBen/AppInfoScanner/releases/latest
国内快速下载通道:
git clone https://gitee.com/kelvin_ben/AppInfoScanner.git
```
2. 安装依赖库
@ -373,6 +378,35 @@ method: 用于配置自动下载过程中需要的请求方法
```
请将错误截图以及对应的APK文件提交至 https://github.com/kelvinBen/AppInfoScanner/issues,作者看到后会及时进行处理。
```
## 自定义规则添加
自定义规则提交路径:
[点击添加自定义规则](https://github.com/kelvinBen/AppInfoScanner/issues/7)
提交格式:
```
1. APP自定义组件添加
如: fastjson的规则如下:
APP组件: fastjson com.alibaba.fastjson
2. 需要进行搜索的字符串
如:查询阿里的AK规则如下:
字符串:
阿里云AK .*accessKeyId.*".*"
3. 需要搜素的web文件后缀名
如:jsp文件的规则如下:
网站: java语言 jsp
4. Android壳规则
如: 某数字公司的壳规则如下:
壳:某数字公司 com.stub.StubApp
```
## 联系作者
@ -383,7 +417,8 @@ method: 用于配置自动下载过程中需要的请求方法
提交需求、提交BUG修复、技术交流、商务合作均可添加作者好友。
## Stargazers
[![Stargazers over time](https://starchart.cc/kelvinBen/AppInfoScanner.svg)](https://starchart.cc/kelvinBen/AppInfoScanner)
# 404StarLink 2.0 - Galaxy
![](https://github.com/knownsec/404StarLink-Project/raw/master/logo.png)

@ -1,3 +1,4 @@
#! /usr/bin/python3
# -*- coding: utf-8 -*-
# Author: kelvinBen
# Github: https://github.com/kelvinBen/AppInfoScanner

@ -1,4 +1,4 @@
#!/usr/bin/env python
#! /usr/bin/python3
# -*- coding: utf-8 -*-
# Author: kelvinBen
# Github: https://github.com/kelvinBen/AppInfoScanner

@ -1,3 +1,4 @@
#! /usr/bin/python3
# -*- coding: utf-8 -*-
# Author: kelvinBen
# Github: https://github.com/kelvinBen/AppInfoScanner
@ -30,7 +31,7 @@ filter_components = [
filter_strs =[
r'https://.*|http://.*',
# r'.*://([[0-9]{1,3}\.]{3}[0-9]{1,3}).*',
r'.*://([\d{1,3}\.]{3}\d{1,3}).*'
r'.*://([\d{1,3}\.]{3}\d{1,3}).*',
# r'/[a-z0-9A-Z]+/.*'
]
@ -73,7 +74,6 @@ shell_list =[
web_file_suffix =[
"html",
"js",
"html",
"xml",
"php",
"jsp",
@ -85,10 +85,10 @@ web_file_suffix =[
# 配置需要忽略网络嗅探的文件后缀名,此处根据具体需求进行配置,默认为不过滤
sniffer_filter=[
# "jpg",
# "png",
# "jpeg",
# "gif",
"jpg",
"png",
"jpeg",
"gif",
]
# 配置自动下载Apk文件或者缓存HTML的请求头信息

@ -1,3 +1,4 @@
#! /usr/bin/python3
# -*- coding: utf-8 -*-
# Author: kelvinBen
# Github: https://github.com/kelvinBen/AppInfoScanner

@ -1,3 +1,4 @@
#! /usr/bin/python3
# -*- coding: utf-8 -*-
# Author: kelvinBen
# Github: https://github.com/kelvinBen/AppInfoScanner
@ -87,7 +88,14 @@ class Bootstrapper(object):
print("[*] Create directory {}".format(out_dir))
if os.path.exists(output_path):
try:
shutil.rmtree(output_path)
except Exception as e:
# 解决windows超长文件名删除问题
if not (platform.system() == "Windows"):
raise e
self.__removed_dirs_cmd__(output_path)
os.makedirs(output_path)
print("[*] Create directory {}".format(output_path))
@ -106,8 +114,15 @@ class Bootstrapper(object):
if os.path.exists(xls_result_path):
os.remove(xls_result_path)
def __removed_dirs_cmd__(self,output_path):
files = os.listdir(output_path)
for file in files:
new_dir = os.path.join(output_path,"newdir")
old_dir = os.path.join(output_path,file)
if not os.path.exists(new_dir):
os.makedirs(new_dir)
os.chdir(output_path)
cmd = ("robocopy %s %s /purge") % (new_dir, old_dir)
os.system(cmd)
os.removedirs(new_dir)
os.removedirs(old_dir)

@ -1,3 +1,4 @@
#! /usr/bin/python3
# -*- coding: utf-8 -*-
# Author: kelvinBen
# Github: https://github.com/kelvinBen/AppInfoScanner
@ -12,7 +13,6 @@ import libs.core as cores
from requests.packages import urllib3
from requests.adapters import HTTPAdapter
class DownloadThreads(threading.Thread):
def __init__(self,input_path,file_name,cache_path,types):
@ -55,7 +55,7 @@ class DownloadThreads(threading.Thread):
sys.stdout.flush()
f.close()
else:
html = resp.html()
html = resp.text
with open(self.cache_path,"w",encoding='utf-8',errors='ignore') as f:
f.write(html)
f.close()

@ -1,3 +1,4 @@
#! /usr/bin/python3
# -*- coding: utf-8 -*-
# Author: kelvinBen
# Github: https://github.com/kelvinBen/AppInfoScanner

@ -1,3 +1,4 @@
#! /usr/bin/python3
# -*- coding: utf-8 -*-
# Author: kelvinBen
# Github: https://github.com/kelvinBen/AppInfoScanner
@ -26,8 +27,6 @@ class ParsesThreads(threading.Thread):
file_path = self.file_queue.get(timeout = 5)
scan_str = ("[+] Scan file : %s" % file_path)
print(scan_str)
if self.types == "iOS":
self.__get_string_by_iOS__(file_path)
else:
@ -41,7 +40,7 @@ class ParsesThreads(threading.Thread):
output_path = cores.output_path
strings_path = cores.strings_path
temp = os.path.join(output_path,"temp.txt")
cmd_str = ("%s %s > %s") % (strings_path,file_path,temp)
cmd_str = ('"%s" "%s" > "%s"') % (str(strings_path),str(file_path),str(temp))
if os.system(cmd_str) == 0:
with open(temp,"r",encoding='utf-8',errors='ignore') as f:
lines = f.readlines()
@ -55,6 +54,13 @@ class ParsesThreads(threading.Thread):
pattern = re.compile(r'\"(.*?)\"')
results = pattern.findall(file_content)
# 搜素AK和SK信息
if not ".js" == file_path[-3:]:
akAndSkList = re.compile(r'.*accessKeyId.*".*"|.*accessKeySecret.*".*"|.*secret.*".*"').findall(file_content)
for akAndSk in akAndSkList:
self.result_list.append(akAndSk.strip())
print("[+] AK or SK in:",akAndSk.strip())
# 遍历所有的字符串
for result in set(results):
self.__parse_string__(result)
@ -64,7 +70,6 @@ class ParsesThreads(threading.Thread):
for filter_str in config.filter_strs:
filter_str_pat = re.compile(filter_str)
filter_resl = filter_str_pat.findall(result)
# 过滤掉未搜索到的内容
if len(filter_resl)!=0:
# 提取第一个结果

@ -1,3 +1,4 @@
#! /usr/bin/python3
# -*- coding: utf-8 -*-
# Author: kelvinBen
# Github: https://github.com/kelvinBen/AppInfoScanner

@ -1,3 +1,4 @@
#! /usr/bin/python3
# -*- coding: utf-8 -*-
# Author: kelvinBen
# Github: https://github.com/kelvinBen/AppInfoScanner
@ -57,6 +58,8 @@ class AndroidTask(object):
dex_md5 = md5_obj.hexdigest().lower()
self.file_identifier.append(dex_md5)
output_path = os.path.join(base_out_path,dex_md5)
if not os.path.exists(output_path):
os.makedirs(output_path)
self.__decode_dex__(file_path,backsmali_path,output_path)
else:
return "error"
@ -73,7 +76,7 @@ class AndroidTask(object):
# 分解apk
def __decode_apk__(self,file_path,apktool_path,output_path):
cmd_str = ("java -jar %s d -f %s -o %s --only-main-classe") % (apktool_path,str(file_path),output_path)
cmd_str = ('java -jar "%s" d -f "%s" -o "%s" --only-main-classe') % (str(apktool_path),str(file_path),str(output_path))
if os.system(cmd_str) == 0:
self.__shell_test__(output_path)
self.__scanner_file_by_apktool__(output_path)
@ -84,7 +87,7 @@ class AndroidTask(object):
# 分解dex
def __decode_dex__(self,file_path,backsmali_path,output_path):
cmd_str = ("java -jar %s d %s") % (backsmali_path,str(file_path))
cmd_str = ('java -jar "%s" d "%s"') % (str(backsmali_path),str(file_path))
if os.system(cmd_str) == 0:
self.__get_scanner_file__(output_path)
else:

@ -1,3 +1,4 @@
#! /usr/bin/python3
# -*- coding: utf-8 -*-
# Author: kelvinBen
# Github: https://github.com/kelvinBen/AppInfoScanner
@ -100,6 +101,7 @@ class BaseTask(object):
def __print_control__(self,packagename,comp_list,file_identifier):
txt_result_path = cores.txt_result_path
xls_result_path = cores.xls_result_path
all_flag = cores.all_flag
if self.sniffer:
print("[*] ========= Sniffing the URL address of the search ===============")
@ -114,11 +116,21 @@ class BaseTask(object):
for json in comp_list:
print(json)
if cores.all_flag:
print("[*] For more information about the search, see TXT file result: %s" %(cores.txt_result_path))
if all_flag:
value_list = []
with open(txt_result_path,"a+",encoding='utf-8',errors='ignore') as f:
for key,value in self.result_dict.items():
f.write(key+"\r")
for result in value:
if result in value_list:
continue
value_list.append(result)
f.write("\t"+result+"\r")
f.close()
print("[*] For more information about the search, see TXT file result: %s" %(txt_result_path))
if self.sniffer:
print("[*] For more information about the search, see XLS file result: %s" %(cores.xls_result_path))
print("[*] For more information about the search, see XLS file result: %s" %(xls_result_path))
def __history_handle__(self):
domain_history_path = cores.domain_history_path

@ -1,3 +1,4 @@
#! /usr/bin/python3
# -*- coding: utf-8 -*-
# Author: kelvinBen
# Github: https://github.com/kelvinBen/AppInfoScanner

@ -1,3 +1,4 @@
#! /usr/bin/python3
# -*- coding: utf-8 -*-
# Author: kelvinBen
# Github: https://github.com/kelvinBen/AppInfoScanner
@ -10,7 +11,6 @@ import platform
import libs.core as cores
from queue import Queue
class iOSTask(object):
elf_file_name = ""
def __init__(self,path):
@ -87,7 +87,6 @@ class iOSTask(object):
with zipfile.ZipFile(self.path,"r") as zip_files:
zip_file_names = zip_files.namelist()
zip_files.extract(zip_file_names[0],output_path)
try:
new_zip_file = zip_file_names[0].encode('cp437').decode('utf-8')
except UnicodeEncodeError:
@ -96,8 +95,8 @@ class iOSTask(object):
old_zip_dir = self.__get_parse_dir__(output_path,zip_file_names[0])
new_zip_dir = self.__get_parse_dir__(output_path,new_zip_file)
os.rename(old_zip_dir,new_zip_dir)
for zip_file in zip_file_names:
for zip_file in zip_file_names:
old_ext_path = zip_files.extract(zip_file,output_path)
start = str(old_ext_path).index("Payload")
dir_path = old_ext_path[start:len(old_ext_path)]
@ -117,6 +116,8 @@ class iOSTask(object):
if not os.path.exists(dir_path):
os.makedirs(dir_path)
shutil.move(old_ext_path, new_ext_path)
if os.path.exists(old_ext_path):
os.remove(old_ext_path)
def __get_parse_dir__(self,output_path,file_path):

@ -1,3 +1,4 @@
#! /usr/bin/python3
# -*- coding: utf-8 -*-
# Author: kelvinBen
# Github: https://github.com/kelvinBen/AppInfoScanner
@ -55,17 +56,11 @@ class NetTask(object):
txt_result_path = cores.txt_result_path
append_file_flag = True
with open(txt_result_path,"a+",encoding='utf-8',errors='ignore') as f:
for key,value in self.result_dict.items():
if cores.all_flag:
f.write(key+"\r")
for result in value:
if result in self.value_list:
continue
self.value_list.append(result)
if cores.all_flag:
f.write("\t"+result+"\r")
if (("http://" in result) or ("https://" in result)) and ("." in result):
domain = result.replace("https://","").replace("http://","")
@ -101,8 +96,6 @@ class NetTask(object):
self.__write_content_in_file__(cores.app_history_path,identifier)
append_file_flag = False
f.close()
def __start_threads__(self,worksheet):
for threadID in range(0,self.threads) :
name = "Thread - " + str(threadID)

@ -1,3 +1,4 @@
#! /usr/bin/python3
# -*- coding: utf-8 -*-
# Author: kelvinBen
# Github: https://github.com/kelvinBen/AppInfoScanner
@ -13,6 +14,7 @@ class WebTask(object):
def __init__(self, path):
self.path = path
self.file_queue = Queue()
self.file_identifier = []
def start(self):
if len(config.web_file_suffix) <=0:
@ -26,7 +28,7 @@ class WebTask(object):
err_info = ("Retrieval of this file type is not supported. Select a file or directory with a suffix of %s" % ",".join(scanner_file_suffix))
raise Exception(err_info)
self.file_queue.put(self.path)
return {"comp_list":[],"shell_flag":False,"file_queue":self.file_queue,"packagename":None}
return {"comp_list":[],"shell_flag":False,"file_queue":self.file_queue,"packagename":None,"file_identifier":self.file_identifier}
def __get_scanner_file__(self,scanner_dir,file_suffix):
dir_or_files = os.listdir(scanner_dir)

Binary file not shown.

Binary file not shown.

@ -1,3 +1,13 @@
### V1.0.8
- 添加AK和SK的检测
- 添加检测规则提交入口
- 添加.gitignore文件
- 优化txt结果集输出方式
- 修复目录中包含空格无法解析的问题
- 修复WEB页面或者目录扫描的问题
- 修复使用默认python执行命令跳出应用商店问题
- 修复iOS应用使用IPA无结果的问题
### V1.0.7
- 新增文件自动下载功能,支持APK文件,非AppStore的IPA文件的下载以及对H5或者html页面进行缓存
- 新增后缀信息修正功能,误填任务类型可以根据后缀名进行自我修正
@ -11,7 +21,6 @@
- 修复以目录的方式对Android反编译原目录被覆盖的问题
- 修复AI分析完毕后不能很好过滤的问题
### V1.0.6
- 新增AI智能分析快速过滤第三方URL地址
- 新增Domain\Title\CDN识别功能
@ -54,7 +63,6 @@
- 新增IPA相关静态扫描规则支持
- 新增对Android加固信息特征识别
### V1.0.0
- 支持目录批量扫描
- 支持DEX、APK、Smali、HTML、JS、等文件的静态资源采集

Loading…
Cancel
Save