You can’t, but you can try to fake it.
Each PE application contains a field in its header that specifies which subsystem it was designed to run under. You can say IMAGE_SUBSYSTEM_WINDOWS_GUI to mark yourself as a Windows GUI application, or you can say IMAGE_SUBSYSTEM_WINDOWS_CUI to say that you are a console application. If you are GUI application, then the program will run without a console.
As he said, you can make your application to run for both console and window mode. In most of the famous applications, which can run under both of these modes will be maintaining two different applications.
Let’s take an example for of devenv. Take the console window (Visual Studio command prompt would be better). Turn on procmon.exe (Process Monitor) and apply proper filter for better viewing. Now give command to console window “devenv /?” (just seeking help from devenv). Now check the process monitor window. You can see the entire system activities to launch the application.
In my understanding this kind of application exploiting alphabetical precedence for .com extension over .exe. So once you give command, Window’s search for the required application would end up in first result that would be devenv.com file.
If you specifically give .exe file. i.e “devenv.exe /?” in the console, it won’t give you console output.
What I’d like to conclude is you will have to maintain two applications to run for console and windows mode.
