I could be mistaken, but I'm under the impression that perhaps the specific instance inside Torghast that you need to enter for this quest can alternate, perhaps each week. The reason that I suspect that this might be the case is because, prior to the weekly reset, when I went to do this quest on my main, the area inside Torghast that was marked on the map for me was The Soulforges, and completing the first layer in Soulforges up to the sixth floor completed the quest for me (ie, the quest item dropped from the last boss there). Also, at the time there were a number of comments here on wowhead stating that you must enter The Soulforges in order to do this quest (at the time there weren't any comments on this page mentioning that you need to do any other instance apart from Soulforges).
However, today (a couple of days after the weekly reset), when I went to do this same quest on a different character, the areas on the map that were indicated for me were not the Soulforges but rather Fracture Chambers and Mort'regar. Now the Soulforges didn't even have a portal that made it possible to enter. I went into Fracture Chambers and completed it (ie, I did up to the sixth floor of layer 1, which was where there was an elite boss)-- and this completed my quest (once again, the quest item dropped from the last boss).
In both runs I needed to complete layer 1 (which has six floors), and in both runs there was an elite boss at the end of the sixth floor. I noticed a few comments on here saying that the boss is presumably found on the first floor, but I suspect that most likely, regardless of which instance you will be prompted to complete, you will probably have to do up to the end of the sixth floor of layer 1. It seems that it's possible that you will get any one of a number of different bosses for the last boss. I think that probably the instances inside Torghast that can be done to complete this quest are just the ones that are open that week, which changes every week.
On here a very helpful user named Regalingual has pointed out that buying blinding smoke capsules from the vendor just before the boss makes it so that the boss can hardly hit you or do any damage to you (please vote Regalingual's helpful comment up!) I have tried this on the boss Custodian Thonar, and I confirm that this worked very well for me: as a cloth-wearer I hardly took any damage while continuously using blinding smoke capsules (I bought as many as I could and just used them one after the next). I think that they last for 30 seconds each, and the cooldown is just 10 seconds. I bought them both from the vendor on the third floor and the one on the sixth floor (I heard that sometimes there isn't a vendor on the sixth floor but I'm not sure if this is true. However, I have had the experience during other runs when neither vendor had any blinding smoke capsules for sale for me at all). The boss that I got at Fracture Chambers was Custodian Thonar; my ilvl was 144 when I did this as a fire mage. In the Soulforges previously, I completed the last boss on my shadow priest (ilvl 142) and did not run into trouble despite not using blinding smoke capsules and not having very good dps, but I think the boss that you get may be random. (Note: the other day I helped a friend complete this same quest, again at Fracture Chambers; this time, even though we got a vendor on both the third and the sixth floor, this time the vendor on the third floor didn't have blinding smoke capsules for sale for me, but it did have them for sale for my friend. The vendor on the sixth floor had them for sale for both of us). This brings up another point: you are able to go into Torghast as a group, so you don't have to go there alone.
Another note: it has been my experience that, in Torghast in general, regardless of the wing that you complete, the boss that you get on the final floor seems to be random. That is, you can get any one of a number of different bosses, and while some of them were not too troublesome for me to solo with my mediocre dps, a few seemed to be disproportionately more difficult. For example, for me and apparently many other people, the boss Warden of Souls was much more difficult than other bosses. But usually I got a final boss that wasn't too terribly bad. (I think that perhaps Warden of Souls can only be a boss at either Mort'regar or Twisting Corridors, but I'm not sure if this is true).
If you have not been to Torghast before and want to know my overall opinion on its difficulty: usually it's not too bad to solo if you complete it on layer 1 (any wing, apart from maybe Twisting Corridors, which is a little different). You can take as long as you want, you can go in a group, usually not too many mobs get pulled at a time, most final bosses aren't too awful (with some exceptions). Depending on the wing, watch out for some platformer game-type traps like swinging axes or flames to jump over, but nothing too terrible (I'm embarrassingly bad at platformer games and I didn't find any of this bad). Any wing you choose will have six floors, apart from Twisting Corridors which has eighteen floors (the final quest in the quest chain will open up access to Twisting Corridors. One note on it: this final quest will ask you to obtain three items, and Twisting Corridors has three main bosses, one after every six floors. However, instead of running eighteen floors in one go, you can run the first six and get the first item; then exit Torghast and start a new run from the beginning, and do the first six floors again and get the second item; and then again for the third item. I'm including this because in Twisting Corridors things get more challenging with every six floors - so if you just exit after the sixth floor and start over again, you'll keep the same difficulty as you had before. Note that if you exit your Torghast run it will not save your progress in Torghast - but it will save the quest item you got from the boss you defeated). Why would anyone bother doing all eighteen floors of Twisting Corridors? Unless I'm mistaken, I think that if you complete all eighteen floors on the highest layer (layer eight?), then you get a mount that can be used in the Maw.