Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

StackOverflow after analyzing dataflow of MZ DOS executable (again) #950

Closed
lempamo opened this issue Oct 12, 2020 · 4 comments
Closed

StackOverflow after analyzing dataflow of MZ DOS executable (again) #950

lempamo opened this issue Oct 12, 2020 · 4 comments

Comments

@lempamo
Copy link

lempamo commented Oct 12, 2020

I seem to have come across the exact same issue that was documented in #850 while I attempted to decompile the DOS version of "The Town With No Name", namely a StackOverflow seemingly caused by an infinite loop somewhere. Reko 0.9.2 only crashed after I canceled that step in decompilation. Please let me know what sort of information you would need to help find the problem and fix it. I'm not very experienced in reverse engineering though, so bear that in mind.

@uxmal
Copy link
Owner

uxmal commented Oct 12, 2020

Thanks for reporting this. The analysis stage of Reko is sensitive to errors happening in earlier stages of the process. Could you provide me with the binary that is causing the problem?

@lempamo
Copy link
Author

lempamo commented Oct 12, 2020

I sent you the file in Gitter DMs.

uxmal added a commit that referenced this issue Oct 12, 2020
@uxmal
Copy link
Owner

uxmal commented Oct 12, 2020

I was able to reproduce the error. The long term fix is not ready for use yet. As a workaround, you can drop the following .dcproject file into the same directory as your EXE file:

<?xml version="1.0" encoding="utf-8"?>
<project xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://schemata.jklnet.org/Reko/v5">
  <arch>x86-real-16</arch>
  <platform>ms-dos</platform>
  <input>
    <filename>DTVPLAY.EXE</filename>
    <asmDir>DTVPLAY.reko</asmDir>
    <srcDir>DTVPLAY.reko</srcDir>
    <includeDir>DTVPLAY.reko</includeDir>
    <resources>DTVPLAY.reko\resources</resources>
    <user>
      <processor />
      <registerValues>
        <assume addr="0800:CD04" reg="ah" value="*" />
        <assume addr="0800:CD1A" reg="ah" value="*" />
      </registerValues>
      <outputFilePolicy>Segment</outputFilePolicy>
    </user>
  </input>
</project>

then open the dcproject file instead of the EXE.

The <registerValues> element tells Reko that when it reaches the addresses 0800:CD04 and 0800:CD1A, it should assume that the register ah has an arbitrary value. Right now, Reko thinks ah is actually 0, which is the MS-DOS int 21h service Terminate Program. This causes cascading errors which end up with the stack overflow.

I've added a new dialog to Reko to let you set these register values using the GUI. Browse to any disassembled instruction, right-click on it and select Register Values. The dialog is crude, but it does the job. Are you able to build Reko yourself? If not, you can get an drop from the CI build on AppVeyor: https://ci.appveyor.com/project/uxmal/reko/builds/35712541/artifacts

Let me know if you still are having stack overflow issues.

@lempamo
Copy link
Author

lempamo commented Oct 15, 2020

I'm not having any more issues with the latest build and the project file you provided, thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants