Computational Intelligence Based Decision Support Tool for Personalized Advertisement Assignment System

In this paper a comprehensive framework that maximizes advertising revenues of a company in a personalized advertisement setting is presented. The research was motivated from a real life problem faced by a company that develops a web based 3-D virtual reality social platform. The objective of the research was both development of a novel business model and the framework. Four heuristics are proposed as part of the framework. The performance of these heuristics was tested with an experimental analysis.


Introduction
The advance of communication technologies has paved the way for a paradigm shift in advertising, namely the personalized advertisement.In the past, mostly mass advertisements were realized under the physical constraints of conventional advertisement media, such as TV's, radios, magazines, open air panels, etc.However, nowadays personalized advertisements are becoming more practical and cheaper particularly through World Wide Web.The virtual environment enables the website owners (i.e., publishers) to display different advertisements to the users (i.e., potential viewers) from their website based on the specifications demanded by their customers (i.e., advertisers), which is a significant advantage of web advertising over conventional ones.
Internet also enables the publishers and/or the advertisers to collect extensive amount of information about the users (therefore the viewers).Personalized advertising makes more sense when information regarding to the users which is buried deep inside this data is brought to surface and utilized.Publishers obtain the information about the users by two means; either provided directly by the users through questionnaires and/or registration forms or mined from the web log of the users online activities.Some of the websites stimulate their users to register and grant further advantages for those who are registered to their website.That's how the web site owners acquire certain demographic data about their users and what they like and/or dislike.On the other hand, the information that can be mined from the online activities of the users from the web logs is also valuable and in some cases more trustworthy than the information acquired directly from the users.The frequency of visiting the website, the duration that they spend on each page, the ads that the user clicks-through, the time of the day the user logins, other users that s/he is in touch with, etc., all provide invaluable information about the user.
This paper was motivated from a real life problem that a local software company faces.The company develops a virtual reality based 3D socialization platform similar to the well known Second Life ‡.The company requires a Personalized Advertisement System ‡ www.secondlife.com(PAS), which handles various constraints that might be imposed by the advertisers regarding to the targeted viewers and allows a flexible pricing scheme.At the core of the PAS lies the Personalized Advertising Problem which has two phases.The first phase is basically a matching problem in which the compatibility of candidate viewers with certain specifications that are set by the advertisers in terms of various features is identified.The second phase is the assignment phase, where a specific advertisement is assigned to an advertisement location whenever the user is in the vicinity of an advertisement location.In this paper a framework for the personalized advertising problem that utilizes computational intelligence during the matching phase and four alternative heuristics during the assignment phase was presented.
The organization of the paper is as follows: Section 2 defines the problem and introduces a novel business model.In Section 3, the relevant literature is reviewed.The proposed solution methods for the matching and the assignment phases are presented in Section 4. In Section 5, the performance of the proposed solution methods are discussed and the paper is finalized with our concluding remarks and further research suggestions in the last section.

Problem Statement
Generally speaking, the virtual reality worlds are real life simulations where users wander around in places which resemble real world places that are modeled by 3D technology.Advertisements that are placed at certain locations within the virtual world are among the major source of income for the companies that are developing and/or maintaining such platforms.The users view certain advertisements while they are online at different virtual locations.For example, while walking down the street occasionally a public bus with an advertisement on the side panels might pass by, or a panel at the bus stop, the building across the road, the mirror in a café, etc. might display an advertisement.
Usually, the advertising revenue received by the company that maintains such platforms is composed of two different sources, namely, the display revenue and the click revenue.The display revenue is realized when a user is "exposed" to an advertisement.Therefore, the event is called exposure.In web browsing a user is categorized as "exposed" whenever the user visits the

Co-published by Atlantis Press and Taylor & Francis
Copyright: the authors web page.On the other hand, in the 3D virtual reality platforms, exposure is defined differently.In such platforms, exposures are realized whenever the user is in the vicinity of a place of an advertisement location.
The definition of vicinity is based on the virtual distance (i.e., magnitude of the advertisement on the screen in terms of the total square pixel).There is no industrial standard and it should be settled between the publisher and advertisers in the contract.Note that depending on the contract terms between the publishers and advertisers time might be an issue as well.That is to say an exposure might be considered to be realized whenever the advertisement occupies a certain total square pixel of the screen for a certain amount of time.
On the other hand, the click revenue is realized whenever the user clicks to an advertisement (and browses to the website of the advertisers), this event is called click-through.The advertisers are seeking effective advertising opportunities that enable them to reach their targeted customers with a smaller budget.On the other hand, the publisher is trying to maximize the total revenue and maintain its attractiveness for the advertisers and the users simultaneously.The users are seeking for a nice environment that they can spend quality time.In this study, our first objective was to develop a business model for the software company that develops and maintains a 3D virtual socialization platform, which plans to attract the advertisers with flexible advertising opportunities including personalized advertisement.For this purpose, various possible alternatives were generated and presented to the company and the final business model was settled together with them.In that sense, the developed business model (i.e., the problem statement) is novel and a contribution to the literature.
According to the developed business model, the publisher (i.e., the company) presents a list of possible features and clustering information associated with each one of the features to the advertisers and the advertisers set the specifications based on their targeted viewers.The features that are provided by the publisher include a set of certain demographics such as age, income level, education level, etc., certain interests such as likes traveling, books, fashion stuff, etc. and certain preferences such as logins mostly at nights, stays short, likes meeting new users, etc.Note that some of the data is nominal, e.g.gender = {Female, Male}, some of them are ordinal, e.g.likes travelling, hates travelling, etc. , and some of them are interval scaled, e.g.age, income level, etc.The advertisers are allowed to specify constraints that are either based on crisp sets, e.g."Advertise this one only to the male users" or fuzzy sets, e.g."Advertise to the users that are young, have middle income and likes travelling a lot".Note that according to the business model, the advertisers will also have the right to specify "don't care" to some of the attributes and unified fuzzy sets such as "young OR middle-aged".
One of the main concerns of the advertisers for exposure of the advertisements is the maximum display number and the minimum display number per individual viewer.The very same advertisement that keeps popping up every corner in a virtual web site can be quite annoying for the viewer, hence not desirable both for the publisher and the advertiser.Furthermore, extensive repetition of the advertisement has no effect after a certain point (i.e.advertisements start to wear out) and there is no need to make payment to the publisher if the message is already received by a particular user.Hence both the annoyance and the law of diminishing returns suggest that setting a maximum display number per person is a common and desirable constraint in practice.
On the other hand, a single exposure might not guarantee that the goal of the advertisement is attained.It is quite possible that the viewer won't pay attention at the first couple of times and totally miss the advertisement, whereas some repetition would enhance the viewer's ability to remember the advertisement in future.Therefore a minimum display number is also a desirable constraint for the advertisers.Note that, whenever the advertiser sets a minimum display number, the publisher would be entitled for a payment after an exposure only if the advertisement is displayed more than it to the specific viewer.
Naturally, there is a maximum payment for every advertiser, i.e., a budget constraint.The contracts would specify the amount of maximum payment which will specify the level that the advertiser would not pay even if the total revenue of the publisher from the exposures and the clicks would exceed that amount.On the other hand the advertisers also desire that the publisher guarantees a certain number of displays so that they ensure to target a critical mass which is required to initiate a word of mouth effect.This constraint is referred to as the minimum payment and the advertiser

Co-published by Atlantis Press and Taylor & Francis
Copyright: the authors makes payment only if the advertisement is exposed to more than a certain number of viewers (i.e., more than a threshold level).In practice it is always possible to set the trivial quantities for all of the above mentioned constraints, that is to say 0 for the minimum display number and minimum payment, and a very large number for maximum display number and the maximum payment depending on the contract between the publisher and the advertiser.On the other hand, exposure is not the only source of income and click through activities are separately priced by the publishers.Click through substantiates that the viewer actually pays attention to and is interested with the advertisement and depending on the situation, such a click through may lead to a real sales at the point.Therefore, whenever a click-through is realized, a separate payment for the click is incurred by the publisher.
The developed business model covers all of the above constraints as well as a flexible pricing scheme which is introduced in order to offer a more satisfactory personalized advertisement opportunity for the advertisers.Also a flexible pricing scheme which enables the advertisers make payments according to the viewers' matching degree (i.e.compatibility score) for their advertisement, is utilized as part of the business model.These compatibility scores are calculated by blending the users' attributes with advertisers' specifications.Actually determination of the compatibility scores is the result of the first phase of the Personalized Advertisement System, namely the matching problem as described earlier.Without loss of generality compatibility score of a user for an advertisement is assumed to be a real number between 0 and 1, where 1 indicates an exact match of the user to the specifications.An interval based approach is developed where the pricing of each interval will be settled through negotiations between the company and the advertiser.For example, in a 4-level pricing scheme, say, the advertiser pays 5¢ for exposures to viewers with compatibility scores in the interval [0.3, 0.7); 8¢ for the interval [0.7, 0.9) and 13¢ for [0.9, 1].In this example the advertiser doesn't pay for viewers that have compatibility score less than 0.3.Finally, in the developed business model, each click through has a constant price which is again determined through negotiations.
After the business model was developed, the company also required a personalized advertisement system (PAS) that will maximize their profit while satisfying the constraints negotiated with the advertisers.The proposed PAS framework had two modules.The first module solved the matching problem (i.e.determination of the compatibility scores) and the second module dealt with the assignment phase, i.e., personalized advertisement assignment problem (PAAS).Before presenting the proposed framework the relevant literature will be discussed next.

Relevant Literature
The personalized advertising problem is becoming relatively more popular only recently, therefore the existing literature on the topic is quiet scarce.A Web of Science search conducted with topics "personalized advertisement" and "optimization" yields no records.Even a search at scholar.google.comyields only 163 hits with the same keywords, most of which are irrelevant to the problem on hand.Furthermore, majority of those that are relevant are U.S. Patents rather than academic papers.However, the virtual reality environments are growing fast; the size of the virtual goods industry reaches to 7.3 Billion USD and the registered users are more than doubled in two years 1 .That is to say, even though academic community seems to be less responsive to the progress of virtual environments, the paradigm shift due to the disappearance of various physical constraints in the fast growing virtual worlds is at the verge.
On the other hand, e-commerce sites such as Amazon, eBay, CdNow are dealing with a problem that is closely related but notably different while recommending their customers alternative products.The systems that are designed with the purpose of advertising the right products to the right user are known as Recommendation Systems (RS).In the literature, these systems are classified in three groups, namely the content based systems, the collaborative filtering systems and hybrid approaches.Content based approaches are designed for advertising a product which is similar to the products that user have shown interest before.Collaborative filtering methods utilize preferences, demographics of the user and advertise a product which similar users are interested before.Hybrid approaches stand in the middle of them.For the

Co-published by Atlantis Press and Taylor & Francis
Copyright: the authors interested readers there are good introductory level papers available in the literature regarding to RS 2 .There are some commonalities with the personalized advertisement systems and the RS's as they both strive to advertise the best advertisements to the users.However, they are different in the sense that, a typical PAS tries to satisfy the requirements of the publishers and the advertisers simultaneously as opposed to the RSs in which the overall objective is to satisfy the will of the publishers.Contracts between the advertisers and the publishers limit the PAS's freedom to always publish the most suitable advertisement for the viewers in order to maximize the overall profit of the publisher.For example, a RS would prefer an advertisement if the viewer's possibility to purchase the product is high, so that a certain amount of commission would be earned by the publisher of the advertisement (i.e., the web site owner).However, the PAS should display an advertisement if it is more profitable, e.g., the corresponding advertiser pays more than the rest.That is to say, the objective function and the constraints imposed due to the contract between the publisher and the advertisers change the structure of the personalized advertisement systems drastically and lead to a new mathematical structure therefore RS's are not viable solutions for personalized advertisement problem.
As previously stated there are two basic problems in PAS, i.e. matching the users to the advertisers and scheduling (assigning) the advertisements to the viewers on real time.Most of the current literature regarding to PAS deals with the matching phase of the problem.Bae et al. propose a web ad selector system based on fuzzy logic that matches the user preferences with the content of the advertisements 3 .Their methodology doesn't include the scheduling constraints such as committed display budget, maximum or minimum display requirements for the advertisements.Also the requirements are not specified by the advertisers but experts decide what type of users should view what type of advertisements hence the scheduling phase is far from handling the needs of the publishers in many real life applications.Later, a methodology that generates the fuzzy rules from historical data collected from user transactions is proposed by Ha 4 .But the scheduling problem is not addressed in this study either.
Kazienko presents a framework for personalized web advertising and addresses both the matching and the scheduling problems 5 .However, the details of the algorithms are not provided in their paper and the paper mostly presents the framework.Zhou et al. also provides the architecture of such a framework without any specific details 6 .The authors leave the scheduling part as future research topic in their paper.Yager presents a targeted e-commerce methodology using fuzzy intelligent agents 7 .The fuzzy reasoning algorithm handles the matching problem is discussed in detail.However, the scheduling problem is briefly mentioned and is based on the bids that will be offered by the intelligent agents of the advertisers.The methodology that determines the bid levels is not addressed in their paper.Kilic proposes a methodology that utilizes fuzzy reasoning for the matching phase and introduces a simple scoring index for the assignment phase 8 .The scoring index is limited and particularly designed to handle only the maximum budget and maximum display constraints.
There are also various assignment and/or scheduling methods in internet advertising that does not include the matching phase.ADWIZ system classifies the users according to the keywords they posted at the website and assigns the advertisements using linear programming approach 9 .The model includes two constraints, namely, the minimum number of exposures associated with the advertisements and the coverage constraint that guarantees that certain portion of the users with specific features are exposed to a particular advertisement.Note that the latter constraint targets the hard matching problem to a degree (e.g. a major car manufacturer can desire that at least %50 of the users who entered a query of "car" on the web site, will be shown their advertisements).
There are some successors of the ADWIZ in the literature.Tomlin demonstrates that the success of ADWIZ system relies highly on the accurate assessment of the model parameters (e.g.click rates) which is not the case in practice 10 .The author concludes that an advertisement selection algorithm must be robust in order to overcome such accuracy problems.Furthermore, Tomlin also modifies the objective that it includes both exposure and click revenues, and proposes a nonlinear programming model of the problem.Nakamura and Abe, further improve the ADWIZ model and render it as a more practical model 11  (iii) Time Interval and developed a heuristic for this problem 12 .Later the problem introduced by Adler et al. is formulated with mathematical programming and a heuristic solution is proposed 13 .In their approach the problem is decomposed into smaller sub problems and the proposed heuristic utilizes the results of these sub problems.Note that, neither of these papers are directly related with our problem since their focus is to select the most efficient set of advertisements to place in an advertising location at a single exposure.At the moment the 3D platform on hand only displays one advertisement in a single advertising location at each exposure, i.e. mixed advertising is not allowed.However, in future studies the above mentioned research 12,13 might be of interest.
The literature on the personalized advertisement systems is still emerging.This is not surprising given the fact that the application areas became more visible only in the past few years.Even though some research have been conducted which are closely related (particularly for the matching phase), the field is still immature and the literature lacks any solution approaches that can be applied to a real life case, such as the problem on hand.

Proposed Methodology
Briefly speaking the objective of PAS is maximizing the total exposure and click-through revenues in a given planning horizon.The advertisers can impose four different constraints, namely, the minimum display number, maximum display number, minimum payment and maximum payment.The payment scheme for the exposures is flexible.In this payment scheme, the advertiser specifies the targeted viewers and based on these specifications, the amount they pay for each exposure to a viewer is calculated with respect to the viewers compatibility score.
The proposed PAS framework had two stages.The first stage determined the compatibility score of the users and the second stage was the advertisement assignment phase which was realized based on the compatibility scores determined in the first stage.In the proposed framework the compatibility scores were determined from the Fuzzy Compatibility Score Calculation Engine (FCSCE) and the second stage was conducted with the Personalized Advertisement Assignment System (PAAS).Figure 1 presents the proposed framework.

Fuzzy compatibility score calculation engine
Let there be C users (i= 1..C, index for number of users) and A advertisements (j= 1..A, index for number of advertisements) in a planning period.The FCSCE utilized fuzzy set theory in order to measure the compatibility score of user i for the advertisement j.Therefore, the first step was fuzzification of the attributes associated with each registered user.This can be achieved by a fuzzy clustering algorithm such as the well known Fuzzy C-Means (FCM) 14 .Note that, according to the proposed business model, the publisher later presents the obtained fuzzy clusters to the advertisers, and the advertisers specify their requirements regarding to the corresponding advertisements based on these clusters.
For each advertisement j, the fuzzy attribute vector F j ={F 1,j , F 2,j ,...,F L,j } (i.e., the requirements), can be specified by the advertiser where L refers to the number of attributes.For example, the advertiser can specify that "The advertisement j should be displayed to the users that are F j = {high income, likes reading a lot, young ...}" and based on the pricing scheme contracted by the parties (i.e., the advertiser and the publisher), the overall compatibility of the viewer entail the corresponding price range, that is to say the revenue from an exposure of the particular advertisement to the viewer.Copyright: the authors 401 In the proposed framework, for each registered user i (denoted as c i ) and advertisement j ( denoted as A j ) combination, the membership degrees for each fuzzy set associated with each attribute of the fuzzy attribute vector of the advertisement j (denoted as F j ) was determined, i.e., )


, based on the resulting fuzzy sets attained from the FCM algorithm that is described earlier.Afterwards, in order to determine the degree that user i met the overall requirements specified by the advertisers for the advertisement j, the individual membership degrees was aggregated over L as follows: where T denotes some t-norm, e.g.min operator.
Finally, based on the  ij values the sets of the customers that satisfy the requirements over a threshold level specified for each advertisement were formed, i.e.,  j , as

Personalized advertisement assignment system
The FCSCE calculates the compatibility scores of each user for each advertisement offline via the Eq. ( 1).These compatibility scores allow us to determine the immediate (i.e., marginal) revenue of exposure of an advertisement to a user based on the pricing scheme.As soon as the user is in the vicinity of an advertisement location, the second stage of the PAS activates, which assigns one of the advertisements to the user.The Personalized Advertisement Assignment System (PAAS) was introduced in order to handle the advertisement assignment problem.Note that, even though the user web log data is a rich source of information regarding to the users past behavior, the future actions such as the logons, exposure, clickthrough are all uncertain.Furthermore, there are typically tens of thousands of registered users of the company.That is to say, the problem is large scale and stochastic in nature.Given the complexity of the problem, heuristic approaches were utilized in PAAS.
Recall that according to the developed business model there were four constraints that the PAAS should take care of, namely the minimum display number, the minimum payment, the maximum display number and the maximum payment constraints.In order to check the criticality of these constraints four heuristics that focus to different subsets of the constraints were developed.
On the other hand, the developed heuristics were closely related with each other.As it will be clear below, the heuristics that handle a larger set of constraints also subsumed the heuristics that deal only a subset of those constraints.
The first algorithm was a greedy algorithm and assigned the most profitable (i.e., highest expected total marginal revenue) advertisement to the viewer whenever the user was in the vicinity of an advertisement location.This heuristic is referred to as the Greedy Algorithm Without Classification (GAWOC).GAWOC was designed to handle the maximum display number constraint and partially the maximum payment constraint.Note that these two constraints are somewhat simpler to handle than the minimum display number and minimum payment constraints since the latter two require a projection of the future user activities.

Algorithm GAWOC
Notation: Q ij = Total number of previous exposures of advertisement j to user i. R j = Total revenue obtained from previous exposures of advertisement j.R ij = Marginal revenue that would be obtained from exposure of advertisement j to user i. TR ij = Expected total marginal revenue that would be obtained from exposing advertisement j to user i CR j = Click revenue that will be obtained from one click of a user for advertisement j Click i = Click rate for user i (calculated based on the historical log data) MaxDisplay j = Maximum display number of advertisement j for each user MaxPay j = Maximum payment for advertisement j (budget limit) 1. Whenever an assignment is required for the user i (i.e., the user is in the vicinity of an advertisement location), calculate the expected total marginal revenue for all advertisements (i.e., TR ij ).Note that the expected total marginal revenue is the sum of the marginal exposure revenue and expected click revenue for that particular advertisement.
Co-published by Atlantis Press and Taylor & Francis Copyright: the authors 2. Assign advertisement j with maximum TR ij to user i.
3. Update Q ij and R j .
The second algorithm was a modified version of the greedy algorithm which was designed to handle the maximum payment and the minimum payment constraints with a probabilistic index and look-ahead ability.It is referred to as the PAASWOC (Personalized Advertisement Assignment System Without Classification).Note that the GAWOC simply assigns the advertisement that has the highest expected total marginal revenue opportunity while checking if the maximum display number and maximum payment constraints.However, the maximum payment constraint cannot be treated sufficiently by means of a simple threshold and requires further attention.Consider a case in which there are two candidate advertisements that might be assigned to a user.Suppose that the specifications (i.e., the requirements set by the advertisers) for the first advertisement are relatively more relaxed so that the set of the potential users with high compatibility scores for this advertisement is larger than the second advertisement which has tighter specifications and thus a limited targeted viewers.Suppose that the expected total marginal revenue for the first advertisement is higher than the second advertisement for the particular user.Even though the first advertisements maximum payment is not exceeded at the time of exposure, it is still possible that assigning the second advertisement might lead to higher overall revenue at the end of the horizon.This is particularly true if the first advertisement would easily reach its maximum payment at the end of the horizon due to high number of potential viewers even if it is not assigned to the particular user, but the second advertisement wouldn't due to its limited potential viewer pool.
In order to handle these situations efficiently, a module referred to as α-Block was introduced in PAASWOC algorithm.In this module the users' log data was utilized and previous periods were simulated based on the realizations of users' activities.For each user, first of all, the advertisements were ranked based on their expected total marginal revenues and their relative revenues were calculated accordingly.Relative revenue for each user refers to the difference between the expected total marginal revenue from the advertisement if that advertisement is exposed to the user and the expected total marginal revenue from the subsequent advertisement (i.e., advertisement with the next highest rank).Next for each advertisement, the users were included into the set of candidate users one at a time based on their relative revenues (in descending order) by an iterative process.
At each iteration of the process, with the current set of the users, the expected revenue that would be obtained for the previous periods for the particular advertisement (based on the realizations of user's activities) was determined.These expected revenues were assumed to be normally distributed with the estimated mean and standard deviation from the realizations of previous periods and the probability of exceeding the maximum payment constraint was calculated.The process of including a new user continued until this probability exceeded the predetermined α value.This routine continued for each one of the advertisement and yielded an associated candidate set of users for them.Whenever a user was in the vicinity of an advertisement location, only those advertisements that included the user in the candidate set were considered during the assignment decision.The pseudo code of α-Block is as follows: Algorithm α-Block Notation: k: 1..A index for the rank of the advertisements m: 1..C index for the rank of users d: 1..D previous representative periods used in simulation α-CS j = α candidate set for advertisement j RR i,j = Relative Revenue of advertisement j for user i RankAd i (j) = Rank of advertisement j for user i wrt. the descending order of TR i,j InvRankAd i (.) = Inverse RankAd function, i.e., InvRankAd i (k) =j if RankAd i (j) = k for user i RankUser j (i) = Rank of user i for advertisement j wrt. the descending order of RR i,j InvRankUser j (.) = Inverse RankUser function, i.e., InvRankUser j (m) =i if RankUser j (i) = m for advertisement j PR j,d = Potential revenue for advertisement j in period d Avr(PR j )=Average of PR j,d over d Stdev(PR j )=Standard deviation of PR j,d over d RevenueObtained j = Revenue obtained for advertisement j until the time of execution SlackMax j = Difference between MaxPay j and revenue obtained from previous exposures, i.e., R j π j = probability that advertisement j will exceed MaxPay j 1-For all i rank TR i,j in descending order and determine RankAd i (j) Note that the α-Block is parametric and the value of α controls the behavior of the module.If α=0, all users would be discarded from the advertisements, else if α=1, all users would remain in the set of all advertisements.
PAASWOC algorithm was also designed to handle the minimum payment constraint which was not considered in GAWOC.Recall that the minimum payment constraint is the minimum acceptable limit of displays for the advertisers in order to ensure that the advertisement reaches to an adequate number of users.Therefore, at any time of the planning horizon, if the probability of exceeding this level for an advertisement turns out to be low, the publisher should consider not publishing this advertisement any more since these exposures might not yield any revenue for the company.In order to deal with this constraint a module referred to as the β-Block was introduced to PAASWOC algorithm.
The β-Block module basically checks the probability of satisfying the minimum payment constraint during the rest of the planning horizon for each advertisement based on the simulations of the earlier periods with the log data of the users.In that sense it is similar to the α-Block.The major difference among the two modules is; in the case of α-Block, the users are discarded or included from the candidate set for the advertisements, however in the case of the β-Block, the advertisements are discarded or included from the candidate set for users based on the simulations of the earlier periods.
In order to attain this goal, for each advertisement, based on the users' log data, previous periods were simulated and total expected revenues were calculated for each period.During this simulation the GAWOC algorithm was utilized, that is to say the advertisements were assigned based on their expected total marginal revenue.Note that, different revenues would be obtained for each period due to the realization of different users' activities.Assuming that these revenues are normally distributed (with estimated mean and variance based on the simulation revenues), the probabilities of not exceeding the minimum payment were calculated and those advertisements that have a probability less than a predetermined β threshold was discarded from the list.The pseudo code of β -Block is as follows: In Algorithm β -Block, steps from 3 to 5 are where the advertisements that seem to be not promising to satisfy the minimum payment constraints were eliminated.Note that after elimination of some advertisements, the chance of satisfying the minimum payment constraints for those that were eliminated earlier, increases (due to the fact that they might be assigned to the users which were originally assigned newly eliminated advertisements in the simulation phase).Steps from 6 to 9 are where some of the eliminated advertisements were included back to the active set of advertisements.Particularly, Step 6 to 8 identifies the order of activation and Step 9 checks if it is possible to activate an eliminated advertisement without abiding the minimum payment constraints of the current active set of advertisements.
The β-Block was also designed to be parametric and β=0 implies that all advertisements remain in the set, and β=1 implies that all advertisements are removed.Note that, during the implementation stage of PAASWOC, the planning horizon is divided into sub periods in order to check if at any time some of the advertisements should be discarded based on the actual realizations observed at that time point.The pseudo code of Algorithm PAASWOC is as follows: Algorithm PAASWOC 1. Apply β -Block and determine SetofActiveAds 2. Apply α-Block with SetofActiveAds and determine α-CS j for j SetofActiveAds 3. Whenever an assignment is required for the user i (i.e., the user is in the vicinity of an advertisement location), calculate the expected total marginal revenue (i.e., TR ij ) for advertisements such that i α-CS j (Recall that the expected total marginal revenue is the sum of the marginal exposure revenue and Co-published by Atlantis Press and Taylor & Francis Copyright: the authors expected click revenue for that particular advertisement) The third and the fourth algorithms, referred to as GA (Greedy Algorithm) and PAAS (Personalized Advertisement Assignment System), respectively, were modified versions of the first two algorithms which aimed to deal with the fourth constraint, namely the minimum display number, as well.Note that the minimum display number constraint imposes that the publisher receives money only if a viewer is exposed to an advertisement more than a certain times.In order to handle this constraint a classification module was introduced which identifies the users that logons less frequent and for shorter durations (referred to as the disloyal users) based on the historical log data.A simple algorithm was developed that decides if the expected exposure for the user is likely to be less than the minimum display numbers of each advertisement.For the users that might not satisfy the minimum display number constraints, the algorithm promoted the advertisements that have a lower minimum display numbers.That is to say, (e.g. for the PAAS Algorithm) only the conditions checked in Step 3 of Algorithm PAASWOC was modified and besides checking if user i is in the α-Candidate Set of advertisement j (i.e., i є α-CS j ), if it is in the same display number cluster with advertisement j was also checked.On the other hand, for the case of Algorithm GA, same condition was introduced to the Step 1 of the Algorithm GAWOC and the rest was same.

Numerical Studies and Discussions
An experimental analysis was conducted in order to test the performance of the proposed heuristics.Part of the data utilized in this analysis was provided by the company and the rest was randomly generated using global industrial standards, experts opinions and academic literature.The data provided by the company was mainly the users log data (logon times, durations, etc.) regarding to the users' activity in previous periods.We generated twelve advertisements for the analysis.The compatibility scores of the users were also randomly generated for each one of the advertisement since the advertisers' requirements were not readily available at the time of the research.We assumed three levels for the pricing scheme of the exposures for the advertisers.The threshold values and corresponding prices were also determined randomly.
The performance of the four algorithms was tested under different experimental conditions with combinations of different levels of maximum payment, minimum payment, maximum display number and minimum display constraints.First of all, the constraints associated with the advertisements were classified either as tight or loose.
According to Novak and Hoffman less than three exposures are ineffective for the user in order to understand the message given 15 .Therefore, we decided that tight minimum display numbers were distributed randomly between 3 and 5 and loose minimum display numbers were distributed between 1 and 3. On the other hand, Novak and Hoffman also state that exposures after 10 exposures have a small incremental effect 15 .Therefore, we decided that tight maximum display numbers were distributed randomly between 6 and 10 and loose maximum display numbers were distributed between 10 and 14.
The minimum and maximum payment amounts were generated via a simulation based on the historical users log data.The tightness was defined as the lower probability of satisfaction of these constraints.For this purpose the average of the expected revenues for the advertisements were calculated based on the simulated results that utilize the greedy algorithm while assigning the advertisements.Later a tight maximum payment constraint value was specified by subtracting a randomly generated multiple of the standard deviation of the expected revenue from the average expected revenue and similarly a loose maximum payment constraint was specified by adding a similar amount to the average expected revenue.Same approach was utilized for the minimum display constraint, however this time the tightness of the constraint referred to the higher value.Note that, the minimum and maximum payment constraints as well as minimum and maximum display number constraints were generated Co-published by Atlantis Press and Taylor & Francis Copyright: the authors independently for every replication during the analysis.The experimental conditions were decided as various combinations of tight and loose constraints.In the discussion part of the analysis, a representation system is utilized to refer to each experimental condition.Cases are coded as (A, B, C, D) in which A refers to the minimum payment constraint, B refers to maximum payment constraints, C refers to minimum display number constraints and D refers to maximum display number constraints.For example, (1, 2, 2, 1) represents the case in which all minimum payments are tight, half of the maximum payments are loose, all minimum display number constraints are loose and all maximum display number constraints are tight.
The company provided 14 days of data and exposures numbers were calculated for every sub period accordingly.The first 7 days of the historical data was utilized in order to estimate the users' activities parameters (i.e. the training data) and the second 7 days was utilized to compare the performances of the algorithms (i.e., the test set).For each of the 36 cases that are tabulated in Table 1 we carried out 10 replications.Furthermore, three different levels of number of users, namely, 100, 250 and 500 was considered in the analysis.Therefore altogether, 36*3*10 = 1080 experiments were conducted.
On the other hand, the α-Block and β-Block introduced earlier were both parametric.Recall that α parameter was introduced in order to decide if the user would be included to α-Candidate Set of the advertisements and β parameter was introduced in order to decide if the advertisement would be eliminated from the active advertisements set.As stated earlier α=1 implies all users would remain in the set of all advertisements and α=0 implies none.Similarly β =0 implies all of the advertisements will remain as active and β=1 implies that all advertisements will be declined from exposure.One can deduce that as α value gets closer to 0 and β value gets closer to 1 then no advertisements will be exposed to the users and as a result the performance of the PAASWOC and PASS which utilizes these parameters would suffer badly.On the other hand, if α is set to be equal to 1 and β is set to be equal to 0 then the advantage of the look ahead feature of these two algorithms disappears.Therefore, our hypothesis was that the best α and β values should be somewhere in between the upper and lower limits (1 and 0 respectively).The experimental analysis was designed to check the validity of this hypothesis and determine the sensitivity of the algorithms with respect to α and β parameters.On the other hand, a steep elimination of users for sure results lower performance.However in case of eliminating an advertisement the situation was blurred since in that case revenue would be generated from the remaining advertisements.Preliminary test results supported this observation and revealed that the performances of the algorithms suffer much faster for lower levels of α parameter.Therefore our focus was shift to only the higher portion of the interval for the α parameter.After this preliminary test stage, six different α values, namely (0.7, 0.8, 0.85, 0.9, 0.95, 1), and six different β values (0, 0.1, 0.3, 0.5, 0.7, 0.9) were determined to be used during the analysis.

Co-published by Atlantis Press and Taylor & Francis
Copyright: the authors Figure 2, depicts the average total revenue of the four algorithms for 500 users across different experimental conditions.Note that the values of PAAS and PAASWOC are the averages resulted from 36 different parameter combinations with ten replications each.The difference among the algorithms becomes significant, that is to say PAAS outperforms GA significantly as the minimum payment constraint becomes tighter.This result was not surprising since PAAS algorithm was designed to handle this constraint as well, by means of the β -Block.
The analysis showed that the difference between the algorithms without classification and with classification (PAASWOC -PAAS and GAWOC-GA) was negligible (statistically not significant).The reason is partly due to the definition of the minimum display number which we adapted from the literature 15 .Note that the right hand tail of the results in Figure 2 that corresponds to the cases with loose minimum payment constraints (i.e., when the effect of minimum payment constraint is minimized) indicated that the all of the algorithms performed similar to each other no matter if the maximum display numbers were tight or loose.This was also expected since all of the algorithms were utilizing the same mechanism to handle the maximum display number constraint.However PAAS slightly outperformed the others when the maximum payment was tighter hinting the influence of the α-Block.
Figure 3 presents the percentage of the best performing α and β values for the PASSWOC algorithm.According to the figure, as α and β values increased the percentage of attaining the best total revenue increased as well.However still for considerably many times the best performance was attained in different parameter values.For example more than 65% of the best results were obtained when α≠1 which demonstrates the effect of the α-Block.Further statistical tests revealed that as number of users increased the difference among the PAAS and GA significantly increased and PAAS significantly outperformed GA.The same thing was also true for

Co-published by Atlantis Press and Taylor & Francis
Copyright: the authors PAASWOC and GAWOC.To sum up, the numerical study and results demonstrated that the proposed improvements to the greedy algorithm, namely the β-Block and the α-Block, improved the performance in terms of monetary returns.Recall that β-Block aimed to manage the minimum payment constraint with a look ahead feature which allowed the system to eliminate the display of unpromising advertisements.On the other hand, the α-Block aimed to manage the maximum payment constraint again with a look ahead feature which allowed the system to eliminate the display of the advertisement to a user which had a larger set of compatible advertisements and preferred a user which had a smaller set.The basic insight of this result is, if the minimum and maximum payment constraints are tight, one should consider using these two blocks in order to attain better revenue.

Future Research and Concluding Remarks
In this paper a relatively new problem was addressed, namely the Personalized Advertisement Problem.It was motivated from a problem of a company that develop a web based 3D virtual reality socialization program.The first contribution of the research is the novel business model introduced that enables the company to present a flexible, personalized advertisement opportunity to the advertisers.Note that, this business model and corresponding problem statement is the first formal introduction of the personalized advertisement system application in a real life context that the authors are aware of.
The second important contribution of the paper was the Personalized Advertisement System Framework that aimed to handle both the matching and the assignment phase of the problem.Note that the proposed framework and the solution approaches are not limited to the use of the companies that develop virtual socialization platforms.With the ongoing introduction of newer technologies and their fast penetration to the daily life of the population, firms in other industries such as the mobile phone companies, internet based service providers, etc. are also looking forward for opportunities of personalized advertisement applications.This paper also hints that, the paradigm shift from brick-mortar to virtual environments would result in a radical change in many other operations management problems.
Since the problem on hand was large scale and stochastic in nature, only heuristic based solutions were considered at this research.However, comparison of the proposed heuristics with various other solution approaches such as math programming is being considered in our future research agenda at least for smaller instances of the problem.

Fig. 1
Fig.1The Personalized Advertisement System Framework ..A index for the rank of the advertisements d: 1..D previous representative periods used in simulation MinPay j = Minimum payment for advertisement j SlackMin j = Difference between revenue obtained from previous exposures, i.e., R j , and MinPay j β -DS = β discard set SetofActiveAds = Subset of all advertisements after β -DS is removed, i.e., {1,2,..,A}\ β -DS PR j,d = Potential revenue for advertisement j in period d Avr(PR j )=Average of PR j,d over d Stdev(PR j )=Standard deviation of PR j,d over d Avr temp (PR j )=Temporary average of PR j,d over d Stdev temp (PR j )= Temporary standard deviation of PR j,d over d TemporaryRevenue j = Expected total revenue if advertisement j β -DS is included to SetofLeftAds RRank(j)= Rank of advertisement j wrt. the descending order of TemporaryRevenue j Co-published by Atlantis Press and Taylor & Francis Copyright: the authors InvRRank(.)=Inverse of RRank(j), i.e., if RRank(j) = k Then InvRRank(k)=j 1. Apply Algorithm GAWOC to the previous D periods and determine the advertisements exposed to the users 2. SlackMin j = R j -MinPay j {Construct β -DS} 3. Initialize β -DS ={} 4. For j=1 to A If SlackMin j ≥ 0 Then go to step 4(i.e., Next j) Else a.For d= 1 to D For i=1 to C If user i was exposed advertisement j during period d as the result of Step 2 Then PR j,d = PR j,d + TR i,j Next i Next d b.Determine Avg(PR j ) and Stdev(PR j ) c. d.If π j < β Then β -DS = β -DS + {j} Next j 5. SetofActiveAds = {1..A}\β -DS {Deletion from β -DS} 6. Initialize TemporaryRevenue j =0, 7.For j=1 to A If j β -DS Then a. SetofActiveAd =SetofActiveAds+{j} b.Apply Algorithm GAWOC to the previous D periods with SetofActiveAds c. Apply Step 4 with SetofActiveAds and determine Avg temp (PR j ) and Stdev temp (PR j ) d.If the minimum payment condition still holds for all of the advertisements in SetofActiveAds (i.e., ,SetofActiveAds π j ≥ β) Then TemporaryRevenue j = Avg temp (PR j )+ R j e. SetofActiveAds = SetofActiveAds\{j} Else Next j 8.For all j rank TemporaryRevenue j in descending order and determine RRank(j) 9.For k=1 to A If TemporaryRevenue j >0 Then a. SetofActiveAds = SetofActiveAds+ InvRRank{k} b.Apply Algorithm GAWOC to the previous D periods with SetofActiveAds c. Apply Step 4 with SetofActiveAds and determine Avg temp (PR j ), Stdev temp (PR j ) and π j d.If the minimum payment condition doesn't hold for any of the advertisements in SetofActiveAds (i.e., π j < β) Then SetofActiveAds = SetofActiveAds\{ InvRRank{k}} Else Next k

Figure 4
Figure 4 is depicted in order to demonstrate how the average performance of the algorithm differs for an instance (when A=2, B=1 and N=100) with different α and β values.Particularly in this case since the maximum payment constraints were all tight the performance of the algorithm significantly deteriorates when α=1.Further statistical tests revealed that as number of users increased the difference among the PAAS and GA significantly increased and PAAS significantly outperformed GA.The same thing was also true for

Fig. 2
Fig. 2 Average total revenue (mean performance) of the four algorithms across different experimental conditions (N=500).

Table 1 .
Table 1 tabulates these combinations.As it is shown in the table the algorithms are tested under 36 distinct cases.The conditions for each constraint used during the experimental analysis.