Help me figure out this encoding?

hummerz5 · 2017-05-16 21:38:33

Hi! I have a project I want to complete but I'm not sure how to go about it. I have this encoding that looks a lot like base64 but isn't.

Looks like from the = that I have three bytes for four characters but idk. Help?

▼pairs

what encoding be this?
edit: tips on how to not fail at this also welcome

LukeM · 2017-05-16 21:58:33

Where did you get this from?
Were you just given that list, or some way to input something and get the output back?

Im pretty sure though that the '='s are some sort of padding, as all the strings end up as a multiple of 4, but I cant see any other patterns atm

hummerz5 · 2017-05-16 22:04:40

Agreed. I have a program that takes input and gives me this, but I don't know what the program is doing. That said, if you have some ideas to test, I can throw 'em in. I think they might be doing this as an odd way to make a string safe for an xml file.
The padding sounds reasonable. For the most part, it appears that information towards the front of the string (as we add to the end) they stay the same. I'm confounded by the exceptions to that rule.

In looking at the individual characters, I'm surprised that the second character only has four things it switches through. I suppose that's indicative of the character dealing with only two bits. Sounds like base64 some more

TaskManager · 2017-05-16 22:17:38

try inputting these and see what comes out:
ah
bg
cf
de

LukeM · 2017-05-16 22:26:25

Well, here is the characters up to h (without padding), from ascii, but should be relatively similar whatever the encoding:

a    og    01101111 01100111
b    oQ    01101111 01010001
c    oA    01101111 01000001
d    pw    01110000 01110111
e    pg    01110000 01100111
f    pQ    01110000 01010001
g    pA    01110000 01000001
h    qw    01110001 01110111

so, what is strange about this is that from the first byte, it seems to be counting up, but from the second byte, it seems to be counting down (using bits 2 and 3)
Maybe the second character has been inverted?

Edit: I have no idea what the last 3 bits are doing though...

Double edit: Im now guessing that it is probably the input encoding that is different, the output looks like normal base64, and up until that strange point it acts like packed bits representing the characters somehow outputed as base64
The strings of 'a's in binary from base64:

101000100000
101000101001000000
101000101001000001000000
101000101001000001000000010110000000
101000101001000001000000010110001110100000
101000101001000001000000010110001110100000111101
101000101001000001000000010110001110100000111101010110000000
110100111000011101011100011001111001111010001011010001101110110001001100110101
100001000101101011110001100010001100110011110000001001011001011100011110001110110000

hummerz5 · 2017-05-16 23:32:32

per maxi's request: I added some binary because you point out interesting binary things

ah    opk=    01100001 01101000    01101111 01110000 01101011 00111101 
bg    oZY=    01100010 01100111    01101111 01011010 01011001 00111101
cf    oJc=    01100011 01100110    01101111 01001010 01100011 00111101 
de    p5Q=    01100100 01100101    01110000 00110101 01010001 00111101

isn't that odd though? I mean if we're really counting backwards in the second part then I wonder if every character does something odd. I wouldnt care to go through the whole system. If it's just going to be layers and layers of idiosyncrasies then I'm disappointed

edit: though we should probably keep in mind the weird table thing

0->63
ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/

opk= -> (40,41,36,0) 101000 101001 101001 000000

Insanity · 2017-05-17 00:22:37

who cares about the encoding

do you even lift brah?

whats ur stats brah?

how much you bench brah?

Bimps

Kira · 2017-05-17 00:27:15

Insanity wrote:

who cares about the encoding
do you even lift brah?

whats ur stats brah?

how much you bench brah?

Their brain muscles are surely stronger than yours

Insanity · 2017-05-17 02:06:20

Kira wrote:

Insanity wrote:
who cares about the encoding
do you even lift brah?

whats ur stats brah?

how much you bench brah?
Their brain muscles are surely stronger than yours

Neurons making FAT gains

AlphaJon · 2017-05-17 22:27:13

I tried fiddling with it, but the output seem to be gibberish, as trying to base64decode them doesn't give any meaningful result, and I didn't find a pattern other than "those 2 close chars produce 2 close values"
I suppose you don't have the source of the program outputting such values.
https://puzzling.stackexchange.com/ or https://stackoverflow.com/ could help (there are rules for cross-posting, though, so don't post on both right away)

TaskManager · 2017-05-17 22:37:40

Can you decode stuff with that or encode only?
Try these, would you:
aa
Aa
aA
AA

Edit: also a string of 8 'a's: aaaaaaaa
You missed that one in your a aa aaa sequence

hummerz5 · 2017-05-17 23:17:59

AlphaJon wrote:

https://puzzling.stackexchange.com/ or https://stackoverflow.com/ could help (there are rules for cross-posting, though, so don't post on both right away)

There's a lot of those. What about CryptoEnthusiasts (or what it's called) or ReverseEngineering? I'm not familiar with them myself. But I wonder if the folks at puzzling might be tired of "what is my encoding"

Agreed, the outputs are weird.

Aa    gpA=
aA    orA=
AA    grA=

aaaaaaaa    W86kiWoeDQQ=

Well part of me thought it could be Base64 with a different lookup table, but that doesn't make sense because the same letters don't repeat when base64-encoding the same string

edit: fwiw, I might end up solving my problem by just applying a macro. It's not clean but my batch is small enough I guess

SirJosh3917 · 2017-05-19 16:04:44

you might want to take a look at this

also i begin working on an html page but then stopped

<DOCTYPE html>
<html>
<head>
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.2.1/jquery.min.js"></script>
<script src="http://peterolson.github.com/BigInteger.js/BigInteger.min.js"></script>
<script>$.baseConvert=function(n,i,t){var r="",e="ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/",o="0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz";if(36>=i&&(n=n.toUpperCase()),i>64||2>t||t>64||2>i?r="Error, selector out of bounds.":n.replace(/[^0-9a-zA-Z+\/]/g,"").length!=n.length?r="Error, invalid character.":$.each((n+"").split(""),function(n,t){i>=63?63==i&&"/"==t&&(r="Error, invalid character."):o.indexOf(t)>=i&&(r="Error, invalid character.")}),""==r){var a=bigInt(0),g="";for(n+="",i+="",t+="",$.each(n.split(""),function(t,r){if(i>=63){var g=bigInt(n.length).minus(t).minus(1),l=bigInt(i).pow(g),d=bigInt(o.indexOf(r)).multiply(l);a=bigInt(a).add(d),console.log(a+" = "+e.indexOf(r)+" * "+l+" ^ "+g)}else{var g=bigInt(n.length).minus(t).minus(1),l=bigInt(i).pow(g),d=bigInt(o.indexOf(r)).multiply(l);a=bigInt(a).add(d),console.log(a+" = "+e.indexOf(r)+" * "+l+" ^ "+g)}});a>0;){g=t>=63?e[bigInt(a).mod(t)]+""+g:o[bigInt(a).mod(t)]+""+g;var l=bigInt(a).mod(t);bigInt(a).minus(l),a=bigInt(a).divide(t)}return g}return r};</script>
</head>
<body>
<noscript><h1>Page requires javascript to work.</h1></noscript>
<p>testing</p>
<script>console.log($.baseConvert("00110011", 2, 64));</script>
</body>
</html>

if you want you could continue it for me

hummerz5 · 2017-05-19 16:08:00

sounds cool. is that like this?

I couldn't get much help from that. But not a bad idea

SirJosh3917 · 2017-05-19 16:12:50

I was saying ( well I didn't say it directly ) that you could've made like a little program that you input your encoded string, and then it brute force checks if that string is equal to any one of the 2-64 encodings

also this might be useful

binary of "aa" in base64

59 57 45 3D

0101 1001 0101 0111 0100 0101 0011 1101

binary of "aa" in this encoding

67 70 41 3D

0110 0111 0111 0000 0100 0001 0011 1101

LukeM · 2017-05-19 17:04:28

We already know that it isnt just a normal baseX
This is because adding a 6th / 7th 'a' changed the values of the first characters, plus a few other things

Also, have you tried decompiling the program you have, or would that be cheating / not possible?

hummerz5

hummerz5 · 2017-05-19 17:14:51

@ninja, good point. I just don't see any patterns. and, as destroyer points out, there's something special after X characters because it completely blows the base idea out of the water

@destroyer
well, I suppose that would be cheating but at any rate, I'm not sure how to go about decompiling .exe or the .app equivalent. I'll take a look... it said it uses .NET packages, but I suppose that doesn't guarantee it uses C# huh

well dotpeek doesn't like it. does that mean it's not .NET?

LukeM

TaskManager · 2017-05-19 17:30:27

destroyer123 wrote:

This is because adding a 6th / 7th 'a' changed the values of the first characters, plus a few other things

maybe it somehow adds up the values of characters and after 6th/7th 'a' the value is big enough to go over a decade/ten/digit/whatever that is?

so we have that upper/lower case thing:

aa    opA=
Aa    gpA=
aA    orA=
AA    grA=

hmm could you do the same for
bb
Bb
bB
BB

cc
Cc
cC
CC

and

zz
Zz
zZ
ZZ

i want to see whether the next characters will keep the same pattern and whether it will change somewhere in the alphabet (thus "z" will have a different pattern)

hummerz5 · 2017-05-19 17:46:19

▼@maxi

but yeah some decompiling guru knowledge would help

▼edit, and because more data is always good

TaskManager · 2017-05-19 20:52:17

okay so, thanks hummerz
for the A B C groups there is a noticeable pattern of "o g o g" that is shared between all of them (plus another pattern separately for each group)
then when it comes to Z group the pattern changes to "u m u m"
i dont know a lot about encrypting/encoding but from what i've heard the further a character is in the alphabet, the higher is it's numeric value, thus Z > A when converted to numbers. however i have no idea how the full order works that includes both upper/lower case characters plus numbers plus special characters (=+- / etc)
now if the values of characters are actually sorted in alphabetical order (z>a)
first we take the A group, the pattern characters are "O" and "G" (15th and 7th in the alphabet respectively, dont know their actual numbers in character table)
then, if we take the Z group, which is supposedly higher than A, the characters are "U" and "M", which are 21st and 13th in the alphabet respectively and they are both 6 characters forward
so i guess throughout the whole sequence from aa to ZZ it goes over a specific digit 6 times. interpret that however you want, idk how to explain and im confused
and most likely the pattern will go like ogogog phphph qiqiqi or ogogog ohohoh phphph or something like that all the way to umumum (try reading out loud all of these)

as for these:

01:88A=
02:88M=
03:88I=
04:88U=
91:+sA=
92:+sM=
93:+sI=
__:nK4=

well yeah they follow the same pattern it seems

i guess that confirms this:

maxi123 wrote:

maybe it somehow adds up the values of characters and after 6th/7th 'a' the value is big enough to go over a decade/ten/digit/whatever that is?

(tbh i dont know whether thats an obvious thing in encrypting stuff because im a noob so i may be just saying things right now that everybody knows already, dont hate)

hummerz, can you decode stuff with that thing or encode only??

SirJosh3917 · 2017-05-19 20:57:25

ok hummerz 1)

use dnSpy, its great for decompiling and editing c#/vb.net/ anything that uses cli and junk
i absolutely love it
https://github.com/0xd4d/dnSpy

2) use snowman for decompiling natively C++ or C things https://derevenets.com/

3) idk

hummerz5 · 2017-05-20 01:01:07

ninjasupeatsninja wrote:

2) use snowman for decompiling natively C++ or C things https://derevenets.com/

I got some interesting things... but there's over a million lines of code and so many redundant incomprehensible casts that I can't read it to save my life

thanks though

maxi123 wrote:

hummerz, can you decode stuff with that thing or encode only??

only encode. You raise some good points about how it appears to cycle, but it's hard to generalize that y'know? like, they follow that cycle but how does it break down? :[

I ended up using a macro to get the particular things I needed, though I'm still awfully curious about this

Nickolai

Swarth100 · 2017-05-20 01:27:04

destroyer123 wrote:

We already know that it isnt just a normal baseX
This is because adding a 6th / 7th 'a' changed the values of the first characters, plus a few other things
Also, have you tried decompiling the program you have, or would that be cheating / not possible?

"Decompiling" code, or rather "disassembling" executables in the case, will always result in unreadable code.
Taking C files as an example, compilation without the -O flag will lead to heavily optimized code (something as simple as a loop could be turned upside down). Labels will have meaningless values and any abstraction above the function level (i.e. classes, interfaces, class extension) will simply not be comprehensible.
Optimised binary files could potentially share no similarity (except behaviour) with the original uncompiled program.

hummerz5 · 2017-05-20 01:32:05

Swarth100 wrote:

Optimised binary files could potentially share no similarity (except behaviour) with the original uncompiled program.

yeah that's pretty much the road I walked. Though, I recognized strings! And a few names of things. Otherwise behavior was a blur

LukeM · 2017-05-20 11:51:59

Swarth100 wrote:

destroyer123 wrote:
We already know that it isnt just a normal baseX
This is because adding a 6th / 7th 'a' changed the values of the first characters, plus a few other things
Also, have you tried decompiling the program you have, or would that be cheating / not possible?
"Decompiling" code, or rather "disassembling" executables in the case, will always result in unreadable code.
Taking C files as an example, compilation without the -O flag will lead to heavily optimized code (something as simple as a loop could be turned upside down). Labels will have meaningless values and any abstraction above the function level (i.e. classes, interfaces, class extension) will simply not be comprehensible.
Optimised binary files could potentially share no similarity (except behaviour) with the original uncompiled program.

Im sure that the inputs / outputs would be distinguishable, so you could still trace through to find out what it is actually doing

Official Everybody Edits Forums

#1 2017-05-16 21:38:33

Help me figure out this encoding?

#2 2017-05-16 21:58:33

Re: Help me figure out this encoding?

#3 2017-05-16 22:04:40, last edited by hummerz5 (2017-05-16 22:05:04)

Re: Help me figure out this encoding?

#4 2017-05-16 22:17:38

Re: Help me figure out this encoding?

#5 2017-05-16 22:26:25, last edited by LukeM (2017-05-16 22:46:20)

Re: Help me figure out this encoding?

#6 2017-05-16 23:32:32, last edited by hummerz5 (2017-05-16 23:41:11)

Re: Help me figure out this encoding?

#7 2017-05-17 00:22:37

Re: Help me figure out this encoding?

#8 2017-05-17 00:27:15

Re: Help me figure out this encoding?

#9 2017-05-17 02:06:20

Re: Help me figure out this encoding?

#10 2017-05-17 22:27:13

Re: Help me figure out this encoding?

#11 2017-05-17 22:37:40, last edited by TaskManager (2017-05-17 22:41:02)

Re: Help me figure out this encoding?

#12 2017-05-17 23:17:59, last edited by hummerz5 (2017-05-18 02:02:04)

Re: Help me figure out this encoding?

#13 2017-05-19 16:04:44

Re: Help me figure out this encoding?

#14 2017-05-19 16:08:00

Re: Help me figure out this encoding?

#15 2017-05-19 16:12:50

Re: Help me figure out this encoding?

#16 2017-05-19 17:04:28, last edited by LukeM (2017-05-19 17:06:15)

Re: Help me figure out this encoding?

#17 2017-05-19 17:14:51, last edited by hummerz5 (2017-05-19 17:25:03)

Re: Help me figure out this encoding?

#18 2017-05-19 17:30:27

Re: Help me figure out this encoding?

#19 2017-05-19 17:46:19, last edited by hummerz5 (2017-05-19 17:47:35)

Re: Help me figure out this encoding?

#20 2017-05-19 20:52:17

Re: Help me figure out this encoding?

#21 2017-05-19 20:57:25

Re: Help me figure out this encoding?

#22 2017-05-20 01:01:07

Re: Help me figure out this encoding?

#23 2017-05-20 01:27:04

Re: Help me figure out this encoding?

#24 2017-05-20 01:32:05

Re: Help me figure out this encoding?

#25 2017-05-20 11:51:59

Re: Help me figure out this encoding?

Board footer