I’ve recently been involved in a large project to deploy Citrix StoreFront for a company with a global presence for 100,000 users. They were using an in-house built front end for their Citrix environment and wished to replace it with the “out of the box” solution. I say “out of the box” because it quickly became highly customised to achieve the results that we required.
This blog post will outline the design desicions as well as the features we enabled and disabled to provide the required experience. I will try and condence the detail down to blog post friendly size but it still might end up being a bit of a read.
Current Environment
The environment consisted of the following consideration factors:
60 XenApp/XenDesktop farms and sites with varying versions
Global userbase with datacentres across continents UK/USA Asia and Africa
A desired support for around 500 login actions per minute per server from customer during high usage or failure scenario’s UK groups need to service more users.
Design
The StoreFront design layout itself is fairly run of the mill for anyone used to it the only difference being theres a heck of a lot of it. The company already utilised F5 LTM and GTM devices so we had to use those for load balancing duties along with various new NetScalers for the AGEE features. We load balanced XML for the XenApp and XenDesktop simplifing and reducing the required StoreFront farm configurations.
![]()
Across the globe there are 7 locations where the StoreFront servers would reside.
2x UK locations
1x US location
3x Asia locations
1x African location
Server virtual hardware sizing
For the virtual hardware design we took the leading practices from Citrix and made two standard hardware specifications based upon the expected usage for the server group.
The small server was given 2vCPU’s and 8GB of RAM which during normal operation would support 500 logins per minute or 30,000 logins per hour.
While the large server was allocated 8vCPU’s and 16GB’s of RAM which would support around 1,300 logins per minute or 80,000 logins per hour.
Based upon the anticipated BAU logins plus failover from F5 GTM if required we required 26 StoreFront servers across the globe and a number of server groups as outlined below.
Now this is of course dependant on the host hardware being performant but during our load testing we comforably got the small servers up to around 2,000 logins per minute before the stress began to show which for us was a great result.
In regards to the subscription database, Citrix recommends allocating a minimum of 5MB disk space plus 8MB for every 1000 application subscriptions. Now these numbers are not something you would normally even consider but due to the number of possible users this suddenly becomes a concern.
Now your milage may vary but we calculated an average of 10 application subscriptions per user which in our largest server group which we anticipated around 58,000 staff this works out as the following:-
58,000 staff x 10 applications = 580,000 subscriptions
580,000 subscriptions / 1000 = 580 8MB chunks
580 chunks x 8MB = 4.640GB
Allocating an additional 4.640GB per virtual machine for the subscription store is not a huge amount in the scheme of things but still worth considering when designing for scale.
Server Groups
To service the users across the globe we utilised several server groups all the server groups were built with a minimum of an N+1 configuration for expected server load and were backed up with NetScalers and F5 GSLB to provide a single url with the desired failover and location awareness features.
The server groups at there largest (which was in the UK) had 6 large servers within the group and the smallest were out in Asia with 2 small servers within the group.
Subscription Synchronisation
However there were two locations near Japan requiring a seperate server group for each location but the staff could log into either one during a normal working day as they had a localised F5 GTM GSLB requirement so this presented a different challenge.
Users logging into lets say Japan1 server group would subscribe to Word and Excel and then logout, but during their next login they might actually hit the Japan2 server group which did not share the same application subscription database so users would have to subscribe to their applications again there as well.
Citrix StoreFront however does allow you to sync the database of an identically named store with one in the in another server group. The article HERE describes how to configure this but it basically boils down to making sure port 808 is open between the servers in the two groups, adding in computer names to the CitrixSubscriptionSyncUsers group in each server and then running a couple of powershell commands.
it is quite important to note that during the initial database sync between server groups the entire database is replicated, however subsequent sync’s are delta files so while the initial sync has a danger of being a good bit of traffic going down the wire that will be it for painful data transer.
XenApp/XenDesktop enumeration
So theres no getting away from it 60 seperate sites and farms is an eye watering amount of infrastructure to try and amalgamate behind StoreFront. The possible load it would place during logon would normally be quite excessive as each time a person logs on the delivery controllers for each farm would be quiered for applications available to the user.
In this instance to resolve and ease the issue we had to utilise two customisations.
User Farm Mapping
XML brokering
By default User Farm Mapping is meant to direct users to particular farms based on user groups and act either as primary and failover type configuration or as a method of presenting a single icon for duplicate applications across farms and sites.
We used UFM for neither of those purposes really just purely for the directing users based on group memberships with a very light smattering of aggregation. This had the advantage of fantastically reducing the query traffic sent to the sites and farms as only the farms the user has a group membership linked to is queried during a logon. (easily verified utilising wireshark or similar tool).
I’ve already written a quite extensive blog post on how to set up and configure user farm mapping which you can view HERE so I’m not going to repeat the basics. However the basic framework of the configuration is as below.
<userFarmMapping name="UserMapping1">
<groups>
<group name="group-hk" sid="*************" />
</groups>
<equivalentFarmSets>
<equivalentFarmSet name="Farm1" loadBalanceMode="LoadBalanced" aggregationGroup="None">
<primaryFarmRefs>
<farm name="XenApp" />
</primaryFarmRefs>
<backupFarmRefs>
</backupFarmRefs>
</equivalentFarmSet>
</equivalentFarmSets>
</userFarmMapping>
<userFarmMapping name="UserMapping2">
<groups>
<group name="group-hk" sid="*************" />
</groups>
<equivalentFarmSets>
<equivalentFarmSet name="Farm2" loadBalanceMode="LoadBalanced" aggregationGroup="None">
<primaryFarmRefs>
<farm name="XenDesktop" />
</primaryFarmRefs>
<backupFarmRefs>
</backupFarmRefs>
</equivalentFarmSet>
</equivalentFarmSets>
</userFarmMapping>
<userFarmMapping name="UserMapping3">
<groups>
<group name="group-uk" sid="*************" />
</groups>
<equivalentFarmSets>
<equivalentFarmSet name="Farm3" loadBalanceMode="LoadBalanced" aggregationGroup="None">
<primaryFarmRefs>
<farm name="XenApp-UK" />
</primaryFarmRefs>
<backupFarmRefs>
</backupFarmRefs>
</equivalentFarmSet>
</equivalentFarmSets>
</userFarmMapping>
So with this snippet of code looking through UserMapping1-3 if I was a user of group-hk when I logged in mapping 1 and 2 would apply and queries to those farms would be sent but not to the farm in mapping 3, as I’m not a member of group-UK.
Its worth noting that within our design from the old infrastructure upto 5 user groups would apply to some of the farms so within our StoreFront configuration we had around 100 seperate user farm mapping code blocks, which during testing there was no noticable performance issues or errors due to the extremely extensive UFM configuration.
StoreFront Server domain mismatch and XML brokering
Another complication of the design is that as the solution was global the StoreFront servers sat in a different domain to those of the majority of the users logging in and backend XenApp/Xendesktop servers.
There is an issue/feature which presents itself when using SSO from within receiver. When logging in with SSO from a desktop your group memberships are gathered from your endpoint and sent through to IIS on StoreFront which is in Domain1 for example. Now if your user groups that you have utilised for user farm mapping or just even application access are domain local groups in Domain2 (where the user account and XA/XD servers are) these are not visable to the StoreFront servers IIS instance which results in no apps being displayed.
This does not occur using standard browser access as the authentication method is different.
In order to resolve this issue the Citrix article outlined here which changes the enumeration of the groups to SID’s from IIS to the Citrix XML service is required.
Automation is key (sort of)
Now for everyones sanity during this project we tried to automate the build process as much as possible however it was troubled slightly by some less than desirable powershell cmdlet’s. As a consultant I make extensive use of the Microsoft Deployment Toolkit (never leave home without it) so we already had the basic StoreFront install in the bag however what it didn’t do was set up the stores or additional farms so I fleshed out the script to include all the farms that we needed to add along with the initial configuration.
The basic install of Citrix StoreFront is the below batch script.
@ECHO OFF
REM Set variables
SET SOURCE=%~dp0
SET SOURCE=%SOURCE:~0,-1%
IF NOT DEFINED LOGDIR SET
LOGDIR=%SystemRoot%\TEMP
ECHO – Installing Citrix StoreFront
START /WAIT CitrixStoreFront-x64.exe
-silent
ECHO – Disable IIS Logging to reduce
excessive logging (enable for troubleshooting)
%windir%\system32\inetsrv\appcmd.exe set
config /section:httpLogging /dontLog:True
Then we switched over to powershell for the rest of the configuration which could be invoked from the batch file by using “powershell.exe -exec bypass -file c:\script\location”.
# Set variables
# SET SOURCE=%~dp0
# SET SOURCE=%SOURCE:~0,-1%
# IF NOT DEFINED LOGDIR SET LOGDIR=%SystemRoot%\TEMP
Echo – Importing Citrix Powershell Modules
cd ‘c:\program files\citrix\receiver storefront\scripts’
./importmodules.ps1
# ./c:\program files\Citrix\Receiver\Scripts\importmodules.ps1
ECHO – Configuring Citrix StoreFront
set-dsinitialconfiguration -hostbaseurl https://LB.company.com -farmname “farm1” -port 80 -transporttype HTTP -sslrelayport 443 -servers @(“xenapp01.company.com”) -loadbalance $false -farmtype XenDesktop -storevirtualpath /Citrix/Company -webreceivervirtualpath /Citrix/CompanyWeb
ECHO – Configuring additional Farms
set-dsxdfarm -farmname “farm2” -servers xenapp02.company.com -port 80 -transporttype HTTP
The set-dsxdfarm option can be replicated as many times as you need to script the addition of the XenDesktop sites. This command however has a MASSIVE issue in StoreFront 3.0.1, there is no way to specify what type of delivery controller you are configuring so running the above command will insert every item in as a XenDesktop controller rather than allowing you to chose XenApp or VIAB (lol). Really hope that gets sorted soon.
As I created the enteries for the script using excel and formula’s it was still way quicker and less error prone to do it this way and then go into the c:\Inetpub\wwwroot\Citrix\Store\Web.conf and alter the farm types manually for the XenApp farms than it was to do the whole thing manually.
What in actual fact we ended up doing was copy the entire farm configuration ( signified by the <farmsets></farmsets> tags)from a completed server group and then paste them into the first server within a server group once the initial install and configuration had been completed. This could then be propagated to the other servers within the farm and configured 60 farms within StoreFront by a simple copy and paste action, not exactly slick but it was quick and easy and repeatable!
Prettification!
Within the environment there were two major use cases for StoreFront one being standard fat client access generally through external or internal access and the second was for thin client access. I say thin client but in actual fact most were running a highly locked down version of Windows on a standard desktop but it did the job. We created two stores for these use cases so each store could be modified to suite the use case.
For the thin clients the client wanted as little option as possible for customisation of clicky stuff so in order to do that we configured the following:-
Full Screen launch
Disable session reconnect
Disabled Subscriptions
Configured SSO (single sign on)
Additionally the GUI of Storefront was highly customised to match the clients look and feel, now this I’ll freely admit was fairly new to me at the start of the project but boy did I pick up a lot in this project. I’m still not a css/js code junky but with Chrome developer mode/notepad and luck on my side I know enough to get by. I’m in the process of creating a blog post that explains a bit more on how you can change the UI of Receiver for Web so hopefully someone will find that useful in the future.
Full Screen Launch
Desktops have the ability to launch within a Window which in a thin client use case is often not desired. You can ensure that the Desktops are always launched in full screen mode by editing the Default.ICA file located in c:\inetpub\wwwroot\StoreName\App_Data\default.ica. Then within the Application section of the file insert the below line.
DesktopViewer-ForceFullScreenStartup=true
Disable Session Reconnect
By default, when staff authenticate to a StoreFront instance any disconnected applications or desktops attempt a reconnect option allowing staff to continue where they left off from before. Within the clients environment this is undesired so must be disabled. This can be configured within the web.conf file located in c:\inetpub\wwwroot\Citrix\StoreName\.
Now search the config file for the allowSessionReconnect=”true” statement and change to false.
Disable Subscriptions
A major part of the StoreFront feature base is the ability to subscribe to your available applications which add’s them to the Favourites folder (also see THIS post on correcting the Americanism’s within StoreFront), for the thin client store the client did not want this. Happily though this is simple and requires no text hacking about in web.conf’s and the like. All you need to do is open up the StoreFront GUI and select the correct store. Then on the right hand window select Disable User Subscriptions.
![subs]()
Configuring Single Sign On
Within the think client use case SSO is pretty much required for every client let alone this one. This required a bit of a major upheaval though as their desktops were still utilising the now deprecated Online plugin version 12 and when you think about the number of staff etc we are touching with this configuration a simple client upgrade suddenly becomes a bit of a task luckily though that is down to the internal IT department to sort out! We advised them that an upgrade would be in order and configured single sign on for a group of test machines in order for the thin clients to login effectively with their local windows credentials. This of course was affected by the XML Broker change we had to make (as mentioned above) in order for remote domain local group memberships to be enumerated correctly on the endpoints. Configuring Single Sign On has not really changed that much in recent history and there are plenty of step by steps to configure that if needs be (See here for the process which talks about it along side Desktop Lock).
Little Critters!
During our design and deployment phase we did come across a bug which menaced us utilising user farm mapping. We worked with Citrix engineering who provided us a hot fix for it. The bug in question presents a Cannot Complete Your Request error when a user logs in to a StoreFront store that has user farm mapping configured for certain circumstances. Additionally within the StoreFronts event viewer a log is created that states “an item with the same key has already been added”
This came about with one or two of the technical users within the pilot phase because they were assigned twice to a static delivery group within a single XenDesktop site. Where normally without user farm mapping the user would be presented with two desktops with the same name (except with the 1/2 monitor ) user farm mapping had a bug where it could not parse the duplicate entries it saw within the responses from the XenDesktop brokers. There is a Bug filed for StoreFront 3.0.1 within Citrix for this (BUG0622730) so if you come across this then refer to that bug and hopefully they’ll send you the private hot fix. I understood that they were looking to implement this in latter versions of StoreFront but I’ve not had the time to test if this has been resolved in StoreFront 3.5.
Summary
I think I’ll leave that here for this post I’ve tried to include the major points of interest from the project in delivering Citrix StoreFront to a large number of user and by and large it was a simple process with the exception of creating the user farm mapping configuration that was extraordinarily tedious but worked very well. Some elements of user farm mapping are now including in the GUI however this will only really suit aggregation or failover of resources for something more complicated you’ll still need to start crafting text configuration and its worth noting that as soon as you do that the StoreFront GUI locks out the ability to manage the controllers, no doubt fearing that someone will change something in the GUI that will mess up the ufm configuration royally.
I was pleasantly surprised how well the virtual hardware withstood the increase until failure test’s which pushed the logon’s per second way past the sizing guidelines given by Citrix and that the extensive user farm mapping did not cause any delay in login actions, however I will be watching the powershell and automation options for StoreFront closely as that did cause us some undesired manual steps which would have been far better if automation was an option.
This blog post isn’t meant to be any sort of design guide for anyone rather than some pointers for the major decisions we made during the design and deployment although this by no means outlines all of the design just some of the more interesting elements. Anyway thanks for reading and as always comments welcome.
Author: Dale Scriven
