granblue-automation-pyautogui

Educational application aimed at automating user-defined workflows for the mobile game, "Granblue Fantasy", using a variety of CV technologies such as PyAutoGUI, GuiBot, and EasyOCR.

This Python application is designed for educational research purposes on studying how to automate certain workflows via image template matching using PyAutoGUI and GuiBot. PyAutoGUI accomplishes this by taking over the mouse, hence why it is recommended to run this on a separate machine than the one that you use daily. This can be circumvented by running this on a virtual machine like VMWare Workstation Player so you can keep using your main computer without interruption.

Screenshots are taken and cropped for PyAutoGUI to perform image template matching. This will determine where the bot is currently at and will inform the bot on what to do next from there. Should PyAutoGUI fail to template match, GuiBot will take over. I have found that on default settings, GuiBot performs admirably well compared to PyAutoGUI in terms of template matching.

There is a feature already in-game that can automate gameplay called "Semi/Full Auto" but does not offer any way to customize what each character does on a turn-by-turn basis. This program’s primary goal is to provide that customization. Users can create their own combat scripts using predefined case-insensitive keywords and can indicate which turns the bot will execute their script, somewhat akin to constructing pseudocode.

For example:

// This is a comment. The bot will ignore this line.
# This is also a comment.

Turn 1:
    // On Turn 1, the following commands will be executed in order:
    // 6th Summon is invoked, character 1 uses Skill 2 and then Skill 4,
    // and finally character 3 uses Skill 3.
    summon(6)
    character1.useSkill(2).useSkill(4)
    character3.useSkill(3)
end

# The bot will keep clicking the Attack button until it reaches the 5th turn.
Turn 5:
    character2.useSkill(2)
end

// Use the exit keyword to leave the raid without retreating.
// Useful when you want to farm multiple raids at once.
exit

Disclaimer

By downloading this program, you consent to your account potentially getting flagged for excessive amounts of farming for multiple hours straight and banned in the next banwave by KMR. I hold no responsibility for how much or how long you use this program for. I trust you have the self-control necessary to only farm in reasonable bursts of time with breaks in between and to always be alert for when the bot encounters the CAPTCHA.

Features

Customize what skills to use during each turn in a user-created plan. Users can select which plan to use when starting the bot.
A launchable GUI to keep track of logs, adjust settings and selecting what combat script to use.
Farm user-defined amounts of specified materials from the supported Farming Modes.
A user-defined timer for how long the bot should run for.
Support for the following game modes:
Quest
Special
Coop
Raid
Event
Guild Wars
Rise of the Beasts
Dread Barrage
Proving Grounds
Xeno Clash
Alert for when anti-bot CAPTCHA pops up.

Requirements

Granblue Fantasy account
Twitter Developer account (optional, needed to farm Raids)
VMware Workstation Player (optional, if your computer is strong enough to support virtualization)

Python Dependencies (make sure to have these installed before moving on to the instructions)

# Execute this command in a terminal opened in the project folder to install all of these requirements.

pip install -r requirements.txt

PyTorch 1.7.1 (Installing the CUDA version will improve the speed of template matching if you have a CUDA-compatible GPU)
GuiBot 0.41.1+
PyAutoGUI 0.9.52+
EasyOCR 1.2.1+
PySide2 5.15.2+
PyYAML 5.3.1+
autopy 4.0.0+
tweepy 3.10.0+
pyclick 0.0.2+

Instructions

Make sure you installed the project dependencies by having Python 3.8.3+ installed and ran pip install -r requirements.txt in a terminal.
Open up the game on a Chromium-based browser and log in if you haven’t already done so. Click away any daily log in popups until you are at the Main/Home screen.

* Make sure the window size is set to this or else the bot will not be able to detect the "Home" button:

correct_window_size

Now open up the program in the following ways for each OS:

For Windows: Open up the terminal in the root of the project folder and type:

python main.py

For Mac: I do not have a Mac system so I do not know how they invoke python. Please look online on how to do that for yourself.

Continue by following the instructions that are now shown to you by heading to the Settings and fill out each section. At the end, you will be notified when the program is ready to begin.

(Optional) Check out the config.ini in the root of the project folder and see what internal settings you would like to change before starting. Some of these settings include mouse settings (Bezier Curve or non-Bezier Curve), Twitter API settings, Dimensional Halo settings, etc.

GitHub

https://github.com/steve1316/granblue-automation-pyautogui

Source: https://pythonawesome.com/granblue-automation-using-template-matching/

Granblue Automation using Template Matching with python