Xmas Spirit

Contents

We get given challenge.py and encrypted.bin. Analysing challenge.py:

It calculates two random values, $a$ and $b$. For every byte $k$ in the plaintext file, it then calculates

And appends the result of that as the encrypted character in encrypted.bin.

Analysis

The plaintext file appears to be letter.pdf, and using this we can work out the values of and because we know the first 4 bytes of every PDF file are %PDF. We can extract the first two bytes of encrypted.bin and compare to the expected two bytes:

Gives us

So we can form two equations here using this information:

We subtract (2) from (1) to get that

And we can multiply both sides by the modular multiplicative inverse of 43, i.e. , which is , to get that

And then we can calculate :

Solution

So now we have the values for and , it's simply a matter of going byte-by-byte and reversing it. I created a simple Sage script to do this with me, and it took a bit of time to run but eventually got the flag.

And the resulting PDF has the flag HTB{4ff1n3_c1ph3r_15_51mpl3_m47h5} within.