Deep Learning Assisted Comic Translation Tool, supports one-click machine translation BallonTranslator

Sep 30, 2024#AI290

AI Translation

This post is translated from Chinese into English through AI.View Original

AI-generated summary

The document describes the "BallonsTranslator" project, which provides a one-click machine translation tool that maintains the original text's formatting, including color, outline, angle, orientation, and alignment. It supports translations between English and Chinese, as well as Japanese and English, with optimized layouts based on extracted background bubbles. The tool includes features for image editing, such as mask editing and repair brushes, as well as rich text editing capabilities, including find and replace functions and Word document import/export. For Windows users, it offers a straightforward setup process that involves downloading a pre-configured package and running a batch file. For those running the source code, it requires Python (version < 3.12) and Git, along with cloning the repository and running a launch script that automatically installs dependencies and downloads necessary models.

Project address: https://github.com/dmMaze/BallonsTranslator

One-click machine translation

The translated text fills back the reference to the original text layout, including color, contour, angle, orientation, alignment, etc. The final effect depends on the overall performance of the four modules: text detection, recognition, erasure, and machine translation. Supports Japanese manga and American comics. English to Chinese and Japanese to English layout has been optimized, with text layout based on the extracted background bubbles. Chinese segmentation is based on pkuseg, and vertical layout for Japanese to Chinese needs improvement.
Image editing
Supports mask editing and repair brush.

Text editing

Supports WYSIWYG rich text editing and some basic layout format adjustments, font style presets. Supports find and replace for full text/source text/translated text. Supports import and export of Word documents. Suitable for comic strips.
Usage instructions
Windows
If you are using Windows and do not want to manually configure the environment, and can access the internet normally: Download BallonsTranslator_dev_src_with_gitpython.7z from MEGA or Google Drive, extract it and run launch_win.bat to start the program. If the libraries and models cannot be downloaded automatically, manually download data and ballontrans_pylibs_win.7z and extract them to the program directory. Run scripts/local_gitpull.bat to get updates.

Running the source code
Install Python < 3.12 (do not use the Microsoft Store version) and Git

# Clone the repository
$ git clone https://github.com/dmMaze/BallonsTranslator.git ; cd BallonsTranslator

# Start the program
$ python3 launch.py

The first run will automatically install dependencies such as torch and download the required models and files. If the model download fails, you need to manually download the data folder from MEGA or Google Drive (or the missing files mentioned in the error) and save it to the corresponding location in the source code directory.