I'm going to start using this news feed as a development blog, to keep people updated with what's happening with upcoming changes, and how development is going in general.
Right now I'm trying to break through a barrier to cycle accurate emulation that no software emulator has done to date. The 68000 processor in the Mega Drive is clocked at around 7.61Mhz, meaning 7,610,000 cycles per second. A single opcode usually takes between 4 and 40 of those cycles to execute, but it can be a lot more than that for MOVEM opcodes, and potentially hundreds for multiply and divide operations. Here's the kicker - the 68000 processor can release the bus to share with other devices, such as for DMA operations from the VDP, or banked memory access from the Z80. The real 68000 processor will respond to bus requests within 2 cycles, or if it's currently actively performing a bus operation itself, at the end of the current bus cycle (4 cycles typically, but can be longer). This means there are around 3,805,000 timing points each second where external requests to obtain the bus need to be checked and responded to in order to get perfect timing accuracy. Right now, Exodus only checks between opcodes, meaning most bus ownership requests are delayed. Most other emulators only synchronise between cores and check for access requests at fixed points, usually thousands of cycles in length. Both are incorrect, and can easily impact timing sensitive code.
There are several challenges in overcoming this barrier. The first is performance. If you want to respond to bus requests accurately within 2 cycles, you need to be checking at 3,805,000 points during execution of your 68000 core for bus access requests each second, and more than that, you need to keep your cores "in sync" so that you don't find out about an access attempt after you've already passed the time when it should have been processed. There is an unavoidable performance hit in doing this. Exodus has a design that aims to minimize that cost, but it will be there. Most of it is already being taken by the platform, hence why Exodus requires more processing power than most (all?) other Mega Drive emulators out there. It was designed to solve these problems, but that means an execution model which has a lot more overhead for synchronisation, because synchronisation is happening a lot more frequently. That's the cost of cycle accurate emulation.
The second barrier is the more annoying one IMO. Your "traditional" emulation core would step through one instruction at a time. If the next instruction is an ADD opcode, you execute it in one go. That's simple, logical. Let's simplify and say it looks something like this:
void AddOpcode(source, target)
{
sourceData = source.read();
targetData = target.read();
result = sourceData + targetData;
target.write(result);
}If you now can grant the bus at between 1 to 13 discrete points within the execution of that single opcode though (for ADD this is correct), what does that look like in code? You can't just call a single linear function for that opcode with one entry and return point anymore. You could try and have blocking calls within the function, so that the execution of the AddOpcode function is suspended until the bus is returned... but then you can't do savestates, because how can you save the current execution state and resume it later when you leave a blocked thread like that? You could try and wait for the processor to be between opcodes for a savestate request to be actioned, but what if you have two processors constantly swapping the bus between each other? They might never reach a point where both are between opcodes and all threads are unblocked.
The best way forward is to turn your opcode execution steps into a kind of state machine, where you can step through the opcode one internal step at a time. That makes the code unfortunately more verbose, and much less branch predictor friendly, so slower, but it looks something like this:
bool AddOpcode(source, target, currentStep)
{
bool done = false;
switch (currentStep)
{
case 0:
sourceData = source.read();
break;
case 1:
targetData = target.read();
break;
case 2:
result = sourceData + targetData;
break;
case 3:
target.write(result);
done = true;
break;
}
++currentStep;
return done;
}This is what I'm currently working on doing for Exodus. This will give truly 100% perfect timing accuracy for the CPU cores, which will be essential when expansions like the MegaCD and 32x come along, which makes the number of bus interactions very complex.
There's actually more to it than this. Part of the work I'm currently doing involves adding perfect handling of DTACK to handle wait cycles during bus access, and group 0 exceptions like bus and address errors, but I'll write more about that at another time.
I'm happy to announce I have now started work on Exodus again. You can expect a fairly steady set of updates over the next few months as I get the repo into shape. No hard plans on what will be in the next release or when that will be, but it'll be coming during the year. I've also made the decision to simplify the licensing model, and as of today Exodus will be governed by the MIT license, a much more permissive licensing model. Feel free to use and adapt the code however you see fit. Any contributions or attribution you want to make back to this project would be greatly appreciated, but not required.
With the accelerating decline of Mercurial support in the industry in general, and the decision by Atlassian to drop Mercurial support from BitBucket in particular, it's past time for me to migrate the Exodus source repository to a new home. Although Exodus hasn't seen much love from me in recent years due to real life keeping me busy, I'm still here in the background biding my time until I can invest in this project again. To give the Exodus code a home until that happens, I've just migrated the repository over to GitHub. Although I have a fondness and personal preference for GitLab, GitHub has become the de-facto standard home for open-source projects over the last decade, and I see no reason to fight that trend. You can now find the Exodus source repository hosted under GitHub at https://github.com/exodusemulator/Exodus. Future development will be based from this location, and the website has been updated to reference this new home.
Exodus 2.1 is now available in the downloads area. Note that you'll need to install the Visual C++ 2017 x64 runtime too if you don't have it, which is available there too. There's quite an impressive list of bugfixes in this release. Job EX-303 in particular fixes a crash that could occur if you had a joystick or gamepad connected, which affected a fair number of people in the previous release. There's also pretty good performance improvements. I measure around a 30% speed improvement overall from Exodus 2.0.1, which is pretty substantial. I've made the VDP plane viewer a but nicer by making the window resizable and making the plane region zoomable, which is nice, but I'm particularly proud of this little nugget:
It's a pixel info dialog you can turn on via "Debug->Mega Drive->VDP->Debug Settings". Just float your cursor over any pixel, and it'll tell you exactly what caused it to appear there. This plays nice with layer removal, so you can peel off a layer at a time and see what's behind it if you want to, and where that pixel came from. It even works for CRAM writes during active scan. Being able to reverse the VDP render pipeline like this was relatively easy in Exodus because of how much info the VDP core holds on to, but it still took a bit of work to pull this off. Give it a spin and let me know what you think. It's great for diagnosing those mystery single line or pixel errors you can get while making something.
Here's the full list of user-facing changes in this release:
Enhancements:
Bug fixes:
This runtime must be installed on your computer in order to run Exodus 2.1 and later. Please download and install the runtime using the link provided here, or download it directly from Microsoft at the following address: http://support.microsoft.com/kb/2019667
This is the official 2.1 release of Exodus. Note that the Visual C++ 2017 x64 runtimes need to be installed in order to run Exodus.
After a long hiatus (over 3 years!), I'm finally returning to give this project some love. At the end of the day, this is just a hobby, and it has to move aside when real life issues require more of my time. Although I can't promise that won't happen again in the future, circumstances have changed, and I do find myself with the time to work on this project again. At this stage I believe this will continue to be the case, and I'll be able to consistently move this emulator forward.
I'm keeping things low key for now, but I'll be putting out a new release very soon, possibly in less than 24 hours. This release will include performance and usability enhancements, a few new features, and various assorted bugfixes. This is mainly a housekeeping release, to clear the decks for bigger work to come.
I've been pondering what I should be focusing on for new development over the last few days, and I feel to stay true to the goals of this project, that has to be accuracy. Although there are a lot of new cool features and enhancements I want to build, Exodus currently falls well short of where it should be in terms of accuracy, and this is primarily down to a lack of sub-opcode level external bus timing accuracy in the 68000 core. While no other Mega Drive emulator does this correctly either, they bend the timings of other system events to work around this limitation and make mainstream games run. I have deliberately avoided doing that in Exodus, and as a result there are numerous games I know of that don't function correctly as a result. This is where I'll be focusing the next development stages, aiming to get 100% cycle accurate M68000 and VDP interaction as a priority. You need to be able to measure accuracy though, so this process will involve hardware testing, and producing test ROMs. By the end of this process, I'll deliver some ROMs which can act as a regression test and a measuring stick, to compare timing accuracy between emulators and the real hardware. I already know roughly how it's going to work, and it's going to be pretty well impossible to bodge, so it'll be interesting to see the results. If you thought OverDrive 2 was brutal on emulators, just wait for this one.....
Contributions to Exodus are welcomed! Please ensure that the code you're contributing fits with the Design Philosophy of Exodus. If you're making a significant modification or addition, it might be worthwhile making contact on the forums first to check if your changes might overlap or be affected by other upcoming changes. The basic steps for making a code submission are as follows:
Any code contributions to the Exodus project are greatly appreciated. This project can only meet its intended goals of supporting a wide variety of platforms through the contributions of others.
The easiest way to contribute code changes to Exodus is to "fork" the Exodus repository, and make the changes you want to make in that fork. When you're happy with the changes you've made, and you'd like them to be merged into the official repository, you submit a "pull request", and your changes can then be reviewed, and merged in once they pass the review. When you hear the word "fork", you may think that implies that you're breaking away from the Exodus project and starting your own independent project that will run forever into the future. This actually isn't how forks are intended to be used with hosted source repositories. The purpose of forks is to keep a relationship between the source repository and your fork of it, so that you can eventually merge the two together again through a pull request. Forks can be thought of as essentially being the same as "branches", but branches designed to work better with independent and disconnected programmers or programming groups.
Here are a few guidelines on how to make working with forks easier, and for your changes to have a better chance of being merged:
A Contributor License Agreement is something that's vital for open source development, and there is an increasing awareness of its importance today. A Contributor License Agreement, or CLA, is designed to protect open source projects, and the contributors to those projects, from possible legal consequences and dilemmas that can and do arise. Without a CLA in place, there are very real liabilities that can have major consequences for contributors and open source projects themselves. Many of these issues are rare, and require malicious intent from someone deliberately trying to harm a project, but there are real life examples of many of these issues arising.
When someone submits code to an open source project without a CLA, the code they submit remains their personal responsibility. When the code is shared or distributed in source or binary form, they remain personally responsible for their contribution. If the code is distributed under some form of license agreement, every contributor of code to that project is entering into a direct and individual agreement with every user of that code. This also means they are personally responsible for any legal ramifications of that code. If the code they have written infringes on any patents, or if any individual alleges that it has infringed on any patents, or if there is a dispute about breach of copyright, or any other form of legal claim of damage or misuse, a legal dispute can be directed at the licensor (the author) by the licensee (the user). This means if an organization or individual had cause to file a legal dispute, they could, and in fact must by law, direct that dispute to individual code contributors to that project. In the event those contributors lost a legal challenge, they would be personally responsible for any damages awarded as a result.
You may assume that when someone submits code changes to an open-source project, they've automatically given permission to the people maintaining that project to actually use the code they sent them, and integrate it in the main source repository, build a compiled program from it, modify it, and so on. Legally they haven't, unless that contributor made a formal statement to that effect. You may also assume that because someone contributes code to an open source project, the person who made the submission has given that project the right to use that code forever into the future. Legally, that also isn't the case unless they stated so. Without a CLA, 10 years after making a submission to a project, a contributor could decide he no longer wants his code to be included in the project, and formally demand it be removed. Legally it would be his right to do so. In this kind of case, his code contributions, and all derived works (things build from or using that code) must be stripped out of the code repository, and purged from its history. All releases of the software to include his code must be removed from distribution. Any sharing of that code as source or binary is no longer permitted. This could literally destroy open source projects if a significant former contributor turned against the project.
There is another liability for open source projects without a CLA, and that is, as described above, when the code is distributed in source or binary form, without a CLA it is actually each author entering an individual license agreement with each user. This also means that if there is a breach of the license agreement, it is the responsibility of each individual author to pursue legal action to enforce the license. Nobody else has the right to do this. Most contributors would not be willing to do this individually, and in the case of authors that are no longer involved in a project, or no longer contactable, this effectively gives anyone the ability to violate the license agreement for any code those authors have submitted without consequence, because nobody except the original authors can enforce the license.
The fundamental issue here is one of copyright law. Copyright law gives the original author of a work the rights to decide who is allowed to copy or alter their work, and the original author has the exclusive right to make those decisions. The original author also has the exclusive responsibility to ensure his work is used within the rights that he has granted to others. This works well where there is only single author for a work, but where an author is contributing their work as part of a larger work that is formed from the contributions of many others, it is essential that the group overseeing that larger work be granted some rights under copyright law. A Contributor License Agreement is the means by which a contributor to a project grants some rights and responsibilities for their contribution to the group managing that project. In any other place where authors publish their own work through some form of formal journal, paper, book, or similar media, they must agree to assign or grant some irrevocable rights to the publisher, or their submission will not be accepted. Open source software projects must have the same requirements if they are to protect themselves and their contributors legally.
The Contributor License Agreement that the Exodus project has adopted is the same license used by Apache and Google to govern contributions to their open source projects. It is unmodified except to reference the Exodus project where appropriate. This agreement in no way affects your rights to do whatever you want with the code you have written. What it does is ensure that once you've submitted code to this project, you can't change your mind about it later, nor are you legally responsible for what the project does with your code afterwards. It also ensures that the project is free to release your contributed code to other people as source and binary releases. Basically, it's formalizing exactly what you probably thought was already implied by submitting code changes to this project.
In order to clarify the intellectual property license granted with Contributions from any person or entity, the Exodus Open Source Project (the "Project") must have a Contributor License Grant ("Grant") on file that has been signed by each Contributor, indicating agreement to the license terms below. This license is for your protection as a Contributor as well as the protection of the Project and the Exodus Open Source Project Leads (the "Project Leads"); it does not change your rights to use your own Contributions for any other purpose.
You accept and agree to the following terms and conditions for Your present and future Contributions submitted to the Project. Except for the license granted herein to the Project Leads and recipients of software distributed by the Project Leads, You reserve all right, title, and interest in and to Your Contributions.
Definitions.
"You" (or "Your") shall mean the copyright owner or legal entity authorized by the copyright owner that is making this Grant. For legal entities, the entity making a Contribution and all other entities that control, are controlled by, or are under common control with that entity are considered to be a single Contributor. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "Contribution" shall mean any original work of authorship, including any modifications or additions to an existing work, that is intentionally submitted by You to the Project Leads for inclusion in, or documentation of, any of the products managed or maintained by the Project Leads (the "Work"). For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Project Leads or their representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Project Leads for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by You as "Not a Contribution."
Grant of Copyright License. Subject to the terms and conditions of this Grant, You hereby grant to the Project Leads and to recipients of software distributed by the Project Leads a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare derivative works of, publicly display, publicly perform, sublicense, and distribute Your Contributions and such derivative works.
Grant of Patent License. Subject to the terms and conditions of this Grant, You hereby grant to the Project Leads and to recipients of software distributed by the Project Leads a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by You that are necessarily infringed by Your Contribution(s) alone or by combination of Your Contribution(s) with the Work to which such Contribution(s) was submitted. If any entity institutes patent litigation against You or any other entity (including a cross-claim or counterclaim in a lawsuit) alleging that your Contribution, or the Work to which you have contributed, constitutes direct or contributory patent infringement, then any patent licenses granted to that entity under this Grant for that Contribution or Work shall terminate as of the date such litigation is filed.
You represent that you are legally entitled to grant the above license. If your employer(s) has rights to intellectual property that you create that includes your Contributions, you represent that you have received permission to make Contributions on behalf of that employer, that your employer has waived such rights for your Contributions to the Project Leads, or that your employer has executed a separate Corporate Contributor License Grant with the Project Leads.
You represent that each of Your Contributions is Your original creation (see section 7 for submissions on behalf of others). You represent that Your Contribution submissions include complete details of any third-party license or other restriction (including, but not limited to, related patents and trademarks) of which you are personally aware and which are associated with any part of Your Contributions.
You are not expected to provide support for Your Contributions, except to the extent You desire to provide support. You may provide support for free, for a fee, or not at all. Unless required by applicable law or agreed to in writing, You provide Your Contributions on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON- INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE.
Should You wish to submit work that is not Your original creation, You may submit it to the Project Leads separately from any Contribution, identifying the complete details of its source and of any license or other restriction (including, but not limited to, related patents, trademarks, and license agreements) of which you are personally aware, and conspicuously marking the work as "Submitted on behalf of a third-party: [named here]".
You agree to notify the Project Leads of any facts or circumstances of which you become aware that would make these representations inaccurate in any respect.
This is the official 2.0.1 release of Exodus. Note that the Visual C++ 2013 x64 runtimes need to be installed in order to run Exodus. The system is pre-configured to load a Mega Drive system on startup, without a TMSS system bios loaded.
If you wish to make a donation to show your appreciation for this project, you can do so here. Your donation may go towards the hosting costs of the website, or equipment or reference hardware to assist in the development of Exodus. It may also go towards a bunch of flowers for my beautiful wife, to say thanks for your support and patience all those nights I stayed up late working on this project.