This project contains modified code derived from PyTorch tutorials -> intermediate sources -> reinforcement_q_learning.py
https://github.com/pytorch/tutorials/blob/main/intermediate_source/reinforcement_q_learning.py
Copyright (c) 2017-2025 Adam Paszke https://github.com/apaszke, Mark Towers https://github.com/pseudo-rnd-thoughts
Copyright (c) 2025-2026 henrymcneill
The original program was designed for active physical motions, whereas minetargeted a stochastic card game. I deleted and rewrote most parts of the initial DQN program, only retaining the general structure and key PyTorch algorithms.
This project is a Python implementation of a deep reinforcement learning model using PyTorch to play Blackjack. Built by adapting an existing Deep Q-Learning algorithm, this project focuses on training an AI capable of handling the stochastic nature of Blackjack and learning optimal decision-making strategies through experience. The deep learning model has as a total of 14 inputs:
- the number of aces the dealer has
- the number of aces the model has
- the sum of the dealer's cards
- the sum of the model's cards
- the numbers of aces left in the deck
- the number of "2" cards in the deck
- the number of "3" cards in the deck
- ... 14.the number of "10" cards in the deck (including Jacks, Queens, and Kings)
It can therefore count cards and now exactly if it is more favourable to play risky or not.
The model can only hit or stand as its two outputs. The next step would be to upgrade the program to let it double and split as well (improving the program immensely).
The following graph represents the model’s win rate while training. The blue line represents the average win rate of the last 20 games, while the orange one represents the last 500 games. As we can observe, the model achieves a win rate of approximatively 40%, getting at 42% at best. These are great results as the model's win rate is very close to the mathematical 42%, who doesn't count cards, could get. Even if an expert player counts cars, he would only get a 44% win rate at maximum.
Remark: counting cards is only really useful for knowing when to double or not. This is how experienced blackjack players get their gains. The win rate doesn't show how good a player is. Unfortunately, the model does not have access to "doubling" as an output yet. To know if counting cards really is being used by the model, we need to introduce the concept of "doubling" in the next program version.
The model is set to play 18000 games in ~18 seconds, for an average of 0.0015 seconds per game (for an apple M4 Laptop). As I don't have a traditional GPU, I am using a CPU to compute.
The following step would be to not randomly generate numbers, but according to how a real deck of cards would (a card that was already played once can't appear a second time). Therefore, the model could learn how to count cards and find techniques to improve its win rate.
After copying the code and downloading the libraries, you can launch the model's training. Then, a real-time graph, similar to the examples above, will appear, showing the model's progression. After a moment, the training will stop. At the end, through the console, we can test the model by making it play individual games. Therefore, we can see what moves (hit or stand) the model would play for a given position.
Download the Python code that is in the repository. Then, we need to download the following libraries: math, random, time (v.1.0.0), matplotlib (v.3.10.1) and PyTorch (v.1.0.2). We would preferably want to use the same versions as listed above.
Licensed under the BSD 3-Clause License - See the LICENSE document