Death Metal Guitar Tone

Actually I was thinking "Mesa Boogie" intuitively when I read the OP's question. Then I tried to look up their gear and only found the Orange Amp, which I don't know too well, but I remember that it's versatile.

Okay, to clear some things up, while the band might be using Orange amps for some applications, it's most likely for the clean settings. There is absolutely no way you can get their rhythm tone from an Orange amp.
The rhythm tone they have is pure, 100 per cent Mesa Boogie Dual Rectifier. I didn't even have to read what amp the band was using to know this, it's quite obvious from the percussive chunk they get on their rhythm tones that it's a Dual Rectifier.
For smoothing out solos, for example, the solo on Selkies:The Endless Obsession as a great example, Paul Waggoner is running an Ibanez Tube Screamer (which particular model, I do not know, but most of the Tube Scream series sound very much similar or even the same) to get more compression and to add mids to further increase sustain.

Dustie also uses a Rectifier, but I believe he uses an EQ pedal to boost instead of an overdrive, but both can work, just depends on the sound you want from your boost really.
Both are using Boss NS-2 Noise Supressor pedals, which work okay for rhythm guitar parts, but for my personal taste, they are very average noise gates at best.

They both used Ibanez S series guitars (various models over the years) equppied with EMG 81 and 85 pickups (the 81 for bridge pickup/rhythm settings and the 85 for neck/lead settings I believe).
From the photos I've seen, the band is using other guitars in the studio (most likely with passive pickups) through Orange amps for their clean tone, because there is no way you can get a Recto with active pickups to sound warm on the clean channel.

The band has now switched to Paul Reed Smith guitars since they are no longer signed with Ibanez, which exact models I don't remember off the top of my head, but I believe they are Custom 24 (perhaps even 22?) models with tremolo.
From Dustie's mouth, both guitarists now use Dimarzio Deactivators, which are Dimarzios attempt at making active pickups in a passive design.
Don't believe the hype, because really, they are just really sterile sounding passives and face it, no passive will sound like an active.
They are extremely good for getting tight as hell rhythm tones, but suck for anything less than high gain, and let me state again (which might annoy any Dimarzio employees should it happen they view this thread), they are NOT going to sound like actives, they are just really tight passives.

So basically, if you're serious about capturing the tones behind bands like Between the Buried and Me, you gotta get the stuff they have.

Now lets talk in terms of what the OP owns.
First off, I have no idea what guitar he is using, so he needs to tell us what he's using.
Basically, you're guitar should preferably be mahongany, and the pickups are tight sound.
Examples of tight pickups are :Seymour Duncan Distortion, Seymour Duncan Full Shred, Seymour Duncan Blackouts, Seymour Duncan Dimebucker (which sounds like absolute turd IMO, but just my opinion), Seymour Duncan Blackouts Metal (possibly the tightest pickup ever made, as well as having the most output ever), Dimarzio Evo, Dimarzio Deactivator, practically every medium high/high output pickup Bareknuckle Pickups make, and of course, EMG 81s/85s.
If you're using anything like a Seymour Duncan JB, forget about it, since they are so loose, you need an extremely tight amp to balance out the looseness of them, because Rectos are not very tight amps, and they generally demand tight pickups to stay tight.

Well, you have a Marshall MG100HDFX (no offense intended, but ditch the amp dude, there is a reason why practically no professional guitarist would be seen dead with one of these) running through a Digitech RP250.
Sure enough, there is a Rectifier amp model on there, just as you would expect on many modeling devices.
One question............ Are you running the RP250 with the cab simulation turned off?
If you have it turned on, it will sound like ass, because you're already running into a guitar cab with guitar speakers.

Ideally, you want to be running a modeling system through full frequency range speakers (through any power amp of your choice).
The reason for this is, amp modeling systems generally produce sounds within a frequency range of 20Hz to 22KHz, sometimes even higher, but guitar speakers are designed to cut off at certain frequencies, with many guitar speakers falling into the 70Hz generally stopping at a ceiling limit of 6KHz at most.
So basically, you can see you're effectively "choking" your amp modeling capabilities of your Digitech RP.
AFAIK, the RP250 doesn't offer the most amazing amp simulations either.
Seriously, go into a guitar shop, test out, say a Line 6 XT perhaps, and put it up against a real Mesa Boogie Rectifier.
Sure, the XT might 'sound' about 95 per cent of the way there, but it has an annoying fizz you need to EQ out and frankly, the pick attack, the dynamics and the complete tube amp feel is not really there.

You're best bet, if you want not only pretty crushing rhythm tones, but also good clean tones (something you can't get with a Mesa Boogie Rectifer) is to get a Line 6 X3 Live, and pair it up with a decent power amp and full range full frequency speaker set.
This option, of course, wont sound quite as good as a real Rectifier, but the X3 does some pretty good amp sims nonetheless , and this versatile recording/effects/amp simulation, will do 1000 times more things than the Recto could ever do and will be cheaper too.
The only thing you'll lose out on, is that the amp sims wont be quite as real as the real thing, but honestly, that's a small price to pay when you consider how much more versatile an X3 Live is.
Of course, if you're jobless or for whatever reason,  can't afford that, then you may still be able to get 70 per cent of the way with your current set up.

With the Rectifer amp model (labeled as "01 Mesa/Boogie® Dual Rectifier™*" on your modeler), if it's at all similar to other amp modelers that model the way the EQ system works on a Recto, I can explain how it works.
Of course, you could always just set the gain to maximum, the treble to full, mids to 0 and bass to full, but if you do that, your tone will sound about as good as the idiot 14 year old kids that go into guitar stores thinking that max treble and bass, minimum mids and full gain sounds good.
It doesn't.
It sounds completely horrible, and in a band mix you wouldn't even be heard with those settings.

Basically, on a real Recto, changing the EQ settings isn't just "lower bass, and bass is now lower, lower mids and mids are now lowered".
No, it's more complicated than that.

Changing the treble has a drastic effect on the tone.
Turn it up, and it means the mid and bass controls become less effective and hence, less present in the overall sound.
Turning it right down, makes it sound pretty smooth, but it completely lacks bite and will not be suitable for rhythm
Changing the mid range affects the picking dynamics and can change the level of compression somewhat.
Decreasing mids makes the bass and treble more dominant in the sound.
DO NO go below 40 per cent mids, especially playing in a band setting, otherwise no one, including yourself, will be able to hear you properly. If you do choose to scoop all mids, you have to turn the volume up more and increase treble and bass just to be heard, but at this stage, it will just sound like a fizzy, disgusting mess, so please, just use some mids and be heard.
You shouldn't be cranking treble and bass on full, because treble is the territory of the drummer with the high hat/crash cymbals etc, and the bass is where the bassist is, and you want to be able to hear everyone and not infringe on their frequency territories.

Bass is affected by the mids and treble controls, and higher treble settings render the bass section less effective.
What you want, is to have enough heaviness without sounding muddy, so you need to constantly tweak the treble, mids and bass until you find a balanced enough sound.
The bass can get muddy very quickly on a Recto if you don't dial it in right, but if you do dial one in right, you are rewarded with absolutely skull crushing tone.

On your amp modeling, for the gain setting, find a setting that lets all the notes in 5/6 string chords still ring out clearly, but when you palm mute it, it makes your trousers flap around.
On most amp modelers, my general experience is to set the gain no higher than about 50 per cent, as they all seem to have more gain than the actual real amp itself.
About 50 per cent gain will give you the clarity to hear notes in chords, and if your technique is good enough, you will be able to play death and thrash metal riffs and sound brutal all day long.
For soloing, either kick in the Tube Screamer effect model, or even better, make a patch with a more mid range orientated amp and boost that with the Tube Screamer, as that will have more cut (the Recto's aren't the best lead guitar amps around I'm afraid).

Also, your EQ settings should be pretty flat on the Marshall itself, possible dip the mids a bit to compensate for the fact Marshall amps are middy in nature.
Run the RP250 through the effects loop and not through the front end of the amp, if you don't already do so.
The reason for this, the preamp will color the sound of the RP, making it sound more digital and less realistic, and running it through the effects loop bypasses the preamp section the amp, running it straight into the power amp of the Marshall resulting in less coloration of the tone.

Sorry if that was a bit long, but hope you learnt something/hope it helped.

I think that's going to be a bit difficult. You'll surely get a cool hi-gain sound with that amp, but it's probably never going to sound like the Orange Amp they're using. Not sure about the "high pitched wail" you're describing ... could be microtonal feedback from the pickups.