{ "meta": { "video_id": "ZKTzfshXw7w", "source": "transcript-api auto/en", "target_language": "none" }, "segments": [ { "start": 8.519, "end": 27.96, "text": "so welcome to this Global Network webinar my name is Alexander lki and I will be moderating this webinar together with my colleagues Sean LEL and Pa Yu um today we have with us our colleague from the UN economic" }, { "start": 27.96, "end": 52.32, "text": "commission for Europe amen kipke and she will be talking about how to unlock the power of large language models for official statistics um in this webinar we will look at the growing impact of large language models like for instance jet" }, { "start": 52.32, "end": 73.52, "text": "GPT or claw within the official statistics Community these advances these Advanced AI models offer great chances to boost productivity and improve user experience but they also bring challenges related to privacy and" }, { "start": 73.52, "end": 92.68, "text": "accuracy that need careful attention so in this webinar you will learn how large language models can be effectively brought into the world of official statistics opening the door to Future Innovations in the this fast changing" }, { "start": 92.68, "end": 112.6, "text": "field our speaker amelina will present the findings of a white paper of the highlevel group for the modernization of official statistics which provides an overview of the effects of large language models and uh we'll share real world" }, { "start": 112.6, "end": 130.759, "text": "examples of how they are being used in various National and international organizations few words on our speaker so amelina is an Associated expert in the statistical management and modernization unit of the" }, { "start": 130.759, "end": 148.92, "text": "UN economic commission for Europe so our Regional Commission uh for Europe she serves as the Secretariat for the applying data science and modern methods Group which again is a part of the high level group for the modernization of" }, { "start": 148.92, "end": 168.72, "text": "official statistics um amelina is also a project manager for Innovative technology initiatives including cloud computing and generative AI for official statistics uh now before I hand over to amelina uh just for you to know so we" }, { "start": 168.72, "end": 185.08, "text": "will first hear the presentation before we will have a Q&A session at the end uh during the presentation feel free to write your comments and questions into the meeting chat that's very welcome so we uh uh so we can see your questions" }, { "start": 185.08, "end": 200.64, "text": "already before the Q&A uh during the Q&A session uh please raise your hand if you want to ask a question and then we can give you the floor and enable your microphone and cameras um as always This Global Network" }, { "start": 200.64, "end": 219.799, "text": "webinar is being recorded and will be made available on the global network of data officers and statisticians at yr.com slun stats and we invite you to continue the discussions on the global Network after this webinar so now I am" }, { "start": 219.799, "end": 238.95899999999997, "text": "uh will hand over to you amelina uh please the floor is yours thank you so much Alexander for kind introduction think you saved me several minutes from doing it myself which is good uh let me share my slides with you" }, { "start": 245.2, "end": 264.12, "text": "might take a while yeah it always takes a little bit until it shows up they show up great so you can see my presentation slides now yeah it's coming up looks good thank you great so first of all thank you very" }, { "start": 264.12, "end": 283.8, "text": "much for the kind opportunity to present our projects around large language models andic statistics and um um yeah so to kick uh to set the stage I think I will introduce a bit more the HG Moos so established in 2010 hlg Moos" }, { "start": 283.8, "end": 297.44, "text": "serves as a global platform for international collaboration with the aim at modernizing statistical processes uh the group enables National statistical organizations to share best practices and collectively address" }, { "start": 297.44, "end": 311.56, "text": "challenges especially those which are related to integrating new technologies into official statistics our mission is mostly to enhance the efficiency accuracy and the relevance of official statistics in an Ever evolving" }, { "start": 311.56, "end": 325.88, "text": "technological landscape especially nowadays and this goal has become even more urgent with the rapid advancement of the generative AI which we will be talking about today so to effectively navigate this rapidly changing" }, { "start": 325.88, "end": 341.56, "text": "environment hgos operates through several modernization groups and these groups explore and Implement Innovations such as artificial intelligence and large language models within the framework of official statistics so each" }, { "start": 341.56, "end": 359.4, "text": "group uh within the hgos focus on different aspects of statistical modernization like advancing statistical methods or integrating new technologies and addressing policy and of course ethical uses and all together" }, { "start": 359.4, "end": 373.639, "text": "uh they form a robust structure for guiding the future of statistics and data driven World why is it important and why I'm mentioning this because the flexible nature of the group helps us to explore emerging topics quickly and" }, { "start": 373.639, "end": 389.16, "text": "efficiently because sometimes when the emerging topic pop up pops up if you are not starting to explore it right away it might be already too late uh to jump in and one of the latest examples in our explanation is a large language models" }, { "start": 389.16, "end": 407.88, "text": "or llms which led to a white paper you can see the the cover of the white paper on the screen um and this white paper evolved into a full scale project on generative AI so uh the white paper was released in" }, { "start": 407.88, "end": 422.16, "text": "December of the last year and uh immediately after that we started to work on the project of generative AI so generative AI project is a continuation of the white paper I need to clar ify it now so there would be no confuse uh" }, { "start": 422.16, "end": 439.199, "text": "confusement during the presentation because I will refer to both of them so our goal uh in this white paper was to understand the power of large language models and how can they modernize uh the way we conduct and disseminate official" }, { "start": 439.199, "end": 452.84, "text": "statistics which ultimately making our work more efficient and accessible and uh in uh today's webinar my focus will be mostly to share the information with we compiled in the white paper along with some current" }, { "start": 452.84, "end": 467.879, "text": "findings from our generative AI projects and the recent survey we conducted on the use of AI in official statistics but before diving into all of this I I think it's important to understand the Technologies at the core of these" }, { "start": 467.879, "end": 485.4, "text": "initiatives so um I assume we have a very broaden audience with different technical background so I suggest we explore what like large language models are and why they're so significant to our work so what exactly are large language" }, { "start": 485.4, "end": 502.28, "text": "models uh this is a screenshot uh from the white paper but uh nothing changed much so I reuse it so large language models are a form of artificial intelligence designed to understand in uh interpret and generate generate" }, { "start": 502.28, "end": 516.76, "text": "humanlike text so they've shown the tremendous potential in transforming tasks such as language translation some mization textt generation and they are becoming more and more relevant to the work we do in official" }, { "start": 516.76, "end": 533.04, "text": "statistics but uh large language models uh for now at least we can see it's a culmination of continuous development and evolution in the field of artificial intelligence and understanding where they fit within the AI landscape help us" }, { "start": 533.04, "end": 548.079, "text": "to understand their capabilities and limitations more so you can see that the base of this whole technology can be called as artificial intelligence which is a broad field focused mostly on creating systems capable of Performing" }, { "start": 548.079, "end": 563.12, "text": "tasks that typically require human intelligence such as problem solving learning reasoning perception and language understanding then there is another dimension which is machine learning that can be called as a subset" }, { "start": 563.12, "end": 577.72, "text": "of AI that enables computers to improve their performance on specific tasks by learning from data without being explicitly programmed so machine is trained to learn itself another subset so it's a further" }, { "start": 577.72, "end": 593.399, "text": "subset of machine learning is deep learning which uses artificial neural networks with multiple layers to automatically discover patterns in large volumes of data which is highly used in tasks mostly in uh image and speech" }, { "start": 593.399, "end": 609.0, "text": "recognition and now we're coming to a topic that's been on everybody's mind for the last several years which is generative AI which is again refers to artificial intelligence broadest systems that can generate new content or data" }, { "start": 609.0, "end": 626.44, "text": "not explicitly derived from existing examples um but um being able to generate the content itself including texts images and even music nowadays but we will get to the use cases a bit later on today so" }, { "start": 626.44, "end": 639.92, "text": "coming to the large language models we can say that it's it's a type of generative AI designed to understand and generate human language by predicting the next word in the sentence based on the patterns it's learned from a large" }, { "start": 639.92, "end": 658.839, "text": "amount of text so a popular example we put there on this image is uh CH GPT but as was mentioned it's Monica CLA and maybe already other GPT likes uh tools but it's also important here to highlight that uh large language models" }, { "start": 658.839, "end": 673.92, "text": "they does doesn't know the information in the way we do like as humans it doesn't have knowledge or understanding the way it works it predicts the next work based on the pattern that it has learned from the data it was train on so" }, { "start": 673.92, "end": 688.72, "text": "it generates responses by using statistical associations between words and phrases but it doesn't really like truly understand the meaning so while it can provide accurate or useful information which mostly is the case it" }, { "start": 688.72, "end": 706.959, "text": "also is potentially making predictions of the next probable word rather than recalling facts from the memory like we do um okay so as as was mentioned the possibilities for using clar language models are very impressive but of course" }, { "start": 706.959, "end": 722.839, "text": "they have their own limits so to start I think many of us are already using large language models in our everyday work without even thinking about it so I would ask you to react now like if you are using GPT Monica or Cloud to help" }, { "start": 722.839, "end": 740.92, "text": "draft emails correct spellings or assist with respond reporting so if you use it for daily routine tasks you can give a reaction here but if you are not reacting I'm sure you are using it so it's fine okay I see a lot of reactions so" }, { "start": 740.92, "end": 756.399, "text": "you are familiar with how it works for daily tasks so large language models helps us at automating routin tasks like inst statistical organization but basically I think it's like any other organization because we all have to" }, { "start": 756.399, "end": 772.48, "text": "draft emails we have to manage reports we have to translate documents and uh this is a very effective way to use the large language models and it can significantly boost our productivity however um using clar" }, { "start": 772.48, "end": 789.079, "text": "language models uh is not perfect because they are not perfect and humor oversight is very much needed and uh I will repeat this point often during this presentation because um human oversight is the key to using AI" }, { "start": 789.079, "end": 804.32, "text": "responsibly uh another important aspect to keep in mind is that large language model separates as a black boxes so they rely on massive data sets which often means that they come with inherent biases if for example the trained data" }, { "start": 804.32, "end": 819.0, "text": "is inaccurate large language model will reproduce this inaccurate data and again their primarily function is to predict Words which means they can produce responses that could sound very convincing but may be inaccurate or even" }, { "start": 819.0, "end": 834.959, "text": "make no sense while sounding again very convincing that's what what we call as hallucinations that's why that's why uh prompt engineering is very important when you work with large language models I will not go here any details because" }, { "start": 834.959, "end": 851.44, "text": "um we have no time for that and but we are working on it with our generative F projects one of the deliverable includes a detailed chapter on prompt engineering so if you're interested in this topic stay tuned for the release of this part" }, { "start": 851.44, "end": 868.16, "text": "um then um what else large language models can help us when we talk about statistics uh it has potential to streamline various stages of statistical production from servey design to data dissemination for example they can" }, { "start": 868.16, "end": 883.0, "text": "suggest survey question validate needed data and generate synthetic data for testing they can assist with metadata management coding language translation all of which are crucial for statistical organizations but despite this" }, { "start": 883.0, "end": 897.519, "text": "efficiencies human supervision again is a key to ensure the quality of the output of large language models um they can also facilitate processes but they are not substitute for human judgment particularly in a" }, { "start": 897.519, "end": 914.04, "text": "task that require the main specific expertise you can train the model but I think we're still a bit far from there um another part is uh that large language models they're changing the way people access information so for example" }, { "start": 914.04, "end": 929.0, "text": "instead of navigating complex databases user can engage in conversational queries making data more accessible which is good however as large language models grow in popularity official statist statistical organization they" }, { "start": 929.0, "end": 948.279, "text": "must adapt to maintain the relevance because while uh the models provide historical data issues with time timelines and quality always can arise so this make it's essential for and soos to offer high quality accurate and timely data" }, { "start": 948.279, "end": 966.959, "text": "that large language models alone cannot guarantee and with this part I want to move to the risks and some of the mitigation measures uh we found out while we were working on the large language model uh white paper so uh" }, { "start": 966.959, "end": 982.44, "text": "while llms offer substantial opportunities that I mentioned briefly before they also POS significant risk that needed careful considerations so let's uh briefly cover the key risk of implementing KMS and statistical" }, { "start": 982.44, "end": 1000.199, "text": "organizations now so the you can see I put it in the VAR Cent so there is no chance stand for you not to see that one of the important consideration is of course ethical consideration ethical obligations extend beyond um uh mayor" }, { "start": 1000.199, "end": 1015.72, "text": "compliance with laws they um truly underpin the trust that statistical organizations must maintain for institutions delivering official statistics it's very important to ensure that public confidence and uphold a" }, { "start": 1015.72, "end": 1033.559, "text": "social license to operate ethical principles like do not harm are found foundational in the use of AI as in everything else so large language models must not be used in ways that cause harm like doesn't matter if intended or" }, { "start": 1033.559, "end": 1049.6, "text": "unintended for example biases I mentioned already in training data can lead to uh discriminatory outputs and it's important to Define clear expected benefits and maintaining a risk register and by doing it it can help mitigate" }, { "start": 1049.6, "end": 1067.039, "text": "these harms and we also need to ensure that large language models are used for defined purposes that it's aligned with organizational needs and that it respects human authority and then fairness non-discrimination safety security" }, { "start": 1067.039, "end": 1082.52, "text": "transparency they are all the key principles to keep in mind when you are working with ledge language model uh next point is accuracy and validation so uh large language models can generate text that may sound very convincing but" }, { "start": 1082.52, "end": 1100.32, "text": "again it isn't always factually correct they uh lacked uh validation mechanism which means human ver verification is very much needed at this step as well so different inaccuracies biases model drift are common challenges we found out" }, { "start": 1100.32, "end": 1115.96, "text": "and retraining models with updated data is very important but sometimes it can be costly that's why it's in a way risk because uh sometimes uh there might be a preference to avoid the retraining to save money but in the end it can result" }, { "start": 1115.96, "end": 1131.919, "text": "in bigger challenges to solve another point is uh privacy secury so I think it's one of the major concern when uh people are talking about implementing large language models uh because there is a chance that uh it can leak" }, { "start": 1131.919, "end": 1147.559, "text": "sensitive information or it can create data poisoning mitigation strategies could include include strict data governance um limiting uh large language access to sensitive data and again human oversight before publishing llm" }, { "start": 1147.559, "end": 1164.08, "text": "generated content and also there's kind of like literacy that you need to know what you can put in the large language model and what you cannot put there next one is legal and copyright issue uh for example if you train uh" }, { "start": 1164.08, "end": 1182.76, "text": "your model on copyrighted material it can lead to Legal liabilities and uh large language generated content um can raise uh certain concerns to mitigate this you can uh adopt content provenance protocols and understand better the" }, { "start": 1182.76, "end": 1199.28, "text": "legal landscape you're operating especially when you rely on external Larch language model providers and the last one uh lack of literacy and misuse so many users may not fully understand large language models limitation leading" }, { "start": 1199.28, "end": 1216.64, "text": "to potential misuse as mentioned for example leaking confidential information or trusting incorrect outputs so here um you should educate and train um constantly to address these issues alongside um providing clear policies to" }, { "start": 1216.64, "end": 1230.76, "text": "guide large language model use and prevent challenging situation so it's important to recognize these basic risks of course there are more and they are rising but I would say that these fives are the Bas basic ones and you should" }, { "start": 1230.76, "end": 1247.2, "text": "recognize them and take appropriate measures if you want to really use large language models effectively while minimizing potential downsides um so uh keeping this all in mind and to better understand the" }, { "start": 1247.2, "end": 1267.039, "text": "landscape we are operating now we decided to conduct a survey during the 7 um 72nd yes Conference of European statisticians earlier this summer we started this survey in uh June and uh we sum up the results in July August so" }, { "start": 1267.039, "end": 1284.799, "text": "it's um uh the freshest results we have for now and so we gathered insights from 36 National and five International organizations worldwide involved in official statistics so this survey provided a in a snapshot um how" }, { "start": 1284.799, "end": 1301.76, "text": "generative AI is being adopted and uh the challenges organizations are facing now and I don't know if you can really see if it's accessible the QR code you see on the screen uh but if not I can post a link in the chat later but anyway" }, { "start": 1301.76, "end": 1317.6, "text": "this QR code will lead you to the full report and infographics um of the survey so let me briefly talk about some results we gained so there is a broad agreement that generative AI will significantly impact efficiency" }, { "start": 1317.6, "end": 1335.48, "text": "particularly in it related areas coding dissemination and communication almost all organizations have started using AI uh with a major Focus for code and it development and data processing however despite this uh" }, { "start": 1335.48, "end": 1351.08, "text": "widespread adoption you can see many organizations still lack of formal policies governing kius and key concerns include data security privacy accuracy and ethical considerations like what we talked on the previous" }, { "start": 1351.08, "end": 1366.72, "text": "slide mitigation strategies such as developing internal policies and providing staff training were highlighted as crucial to addressing these issues and interestingly only 5% of organizations reported not using AI" }, { "start": 1366.72, "end": 1382.2, "text": "at all which in our opinion underscores the Technologies growing importance and as I move forward I will share some of the most compelling use cases highlighting how organizations are already leveraging AI across different" }, { "start": 1382.2, "end": 1397.36, "text": "application areas and when I say AI here I mean mostly generative AI That's large language models in this case because machine learning for example or de learning uh is something else and we were not considering it while we were" }, { "start": 1397.36, "end": 1413.64, "text": "compiling the survey or creating white paper for example so while we are focusing on several key areas uh today I want to knowe that uh the potential applications of generative Ai and official statistics of course extends" }, { "start": 1413.64, "end": 1432.799, "text": "far beyond the example we would talk about so let's start uh with um use cases and data processing and Analysis um you can see uh there are the like stars actually I Googled how is the what is the correct name of this symol" }, { "start": 1432.799, "end": 1451.6, "text": "in English and it was something else but I also Googled if I can call it a star and yes I can can so you can see Star here uh these are the use cases uh covered in a greater detail uh in the large language uh models white paper uh" }, { "start": 1451.6, "end": 1469.44, "text": "so there you can find sections on business problems value addition results and uh HR requirements quality metrics stakeholder responses barriers risks and mitigations on the specific use cases so if you want to know more please refer to" }, { "start": 1469.44, "end": 1489.24, "text": "the white paper and also we have compiled the use case repository accessible via QR code or again if you it's if it's not accessible for you I can post a link later um we have compile 23 individual use cases um there um" }, { "start": 1489.24, "end": 1504.6, "text": "which you can also read more and connect with the author if you are interested in more technical details okay so uh the first application area I want to cover now is data processing and Analysis according to our" }, { "start": 1504.6, "end": 1523.72, "text": "survey 46 um percent of respondents are using AI for tasks like data editing synthetic data generation classification and 39% use it for analytical purposes like report generation as example uh Bank of" }, { "start": 1523.72, "end": 1538.919, "text": "international settlements has developed AI tools for metadata editing which automate tasks like correcting spelling formatting and grammar checking and according to them uh this not only saves time for them but also enhances" }, { "start": 1538.919, "end": 1556.6, "text": "consistency and accuracy across databases and uh the latest update I got from them that they are thinking about how they can train like um um different models to communicate with each other excluding the human supervision like at" }, { "start": 1556.6, "end": 1571.24, "text": "this stage another area of application is statistical classification uh here we have the example of the Australian Bureau of Statistics uh who used large language models to update occupation codes and" }, { "start": 1571.24, "end": 1588.919, "text": "according to their experiments they achieved uh 70% quality rate compared to human efforts and they saved over 1,600 working hours um they also presented us uh the test they run and uh majority of the" }, { "start": 1588.919, "end": 1606.48, "text": "expert they couldn't tell uh What uh occupation code was created by the human and what occupation code was created by llm and um their last update from them that they're participating in their Australian national award for" }, { "start": 1606.48, "end": 1624.52, "text": "Innovations uh the results will be run in mid November so fingers crossed uh this use case will win but I think it's a indication of a clear success of the application of large language model in this area next one is synthetic data" }, { "start": 1624.52, "end": 1639.76, "text": "generation uh which is also the area of growing importance so llms can generate data that fulfill specific characteristic like creating balanced samples for machine learning models and according to the survey several" }, { "start": 1639.76, "end": 1658.399, "text": "statistical organizations uh actively exploring this potential we don't have compiling use cases for now but were like were trying to be updated on their developments in this area then uh in this um in data processing and Analysis" }, { "start": 1658.399, "end": 1675.6, "text": "area there is U important use case on report generation we have example of uh bundes bundes Bank in Germany they are developing the text based intelligent assistant platform I think they developed it already it's for uh" }, { "start": 1675.6, "end": 1692.24, "text": "internal use and they use various versions of chat GPT to generate reports in both English and German uh similarly statistics Canada is experimenting with AI to draft reports from sensus data creating detailed articles that uh of" }, { "start": 1692.24, "end": 1710.279, "text": "course they then reviewed for accuracy because it sensors data so they it should be perfect uh next one is um code and it development according to our service 71% of respondents are using or experimenting at least with AI for" }, { "start": 1710.279, "end": 1728.279, "text": "coding tasks this includes code generation translation and assisting encoded coding processes and one notable example we have from Central Statistics Office of Ireland uh they develop the SAS to our trans complier using open AI" }, { "start": 1728.279, "end": 1743.039, "text": "API with this tool they automate the translation of SAS code to R which significantly reducing the time and effort requiring to transition between this two programming languages and this application is already in production and" }, { "start": 1743.039, "end": 1760.08, "text": "they has been open sourced it on the GitHub so it allows others to benefit from this Innovative solution and as we know uh the similar tool has been developed at Inc France so this example in uh code translation illustrates how" }, { "start": 1760.08, "end": 1775.0, "text": "AI can play an important role in modernizing it operations within statistical organizations which can make uh the process more efficient and also enable faster adaptations to these new technologies and uh the last area I want" }, { "start": 1775.0, "end": 1790.559, "text": "to cover but of course not the least is communication uh this is for example where my personal Journey with AI began as I explore how generative AI could help us improve the communication of official statistics so AI is helping to" }, { "start": 1790.559, "end": 1809.96, "text": "create a clearer more engaging content which is normally a Cornerstone uh for maintaining public public trust and transparency which we are aiming to for example in our statistics division at UNC um we use uh large language models" }, { "start": 1809.96, "end": 1829.519, "text": "to craft social media posts and we try to ensure consistency in tone and style for example I train to model with a manually written written by me posts that I reviewed and um uh now if I uh want to WR write the post very fast I" }, { "start": 1829.519, "end": 1847.24, "text": "just put the basic information to the model I wrote and it gives me the already ready to go post more or less in the style as I would write it so with this it helps to maintain the coherence in our messaging and also using AI like" }, { "start": 1847.24, "end": 1862.24, "text": "we are trying to use AI to generate visuals that be aligned with our let's call it brand identity uh you can see the QR code here and another QR code I don't know if you can access it again or not apologies for" }, { "start": 1862.24, "end": 1879.76, "text": "that but this is a link to our uh LinkedIn page where you can see that the post and the AI images are mostly produced with the help of generative Ai and as we know statistics Indonesia is exploring similar approach" }, { "start": 1879.76, "end": 1896.519, "text": "with visuals as we do and apart from visuals AI is also being used to create other non textual contents such as audio and videos uh we don't have a lot of examples in this area but we have an example from statistics Canada uh who" }, { "start": 1896.519, "end": 1913.6, "text": "are exploring AI dreaming approaches to generate bilingual content with uh narration and visuals and they try to en enhance both the reach and accessibility of their data so that this demonstrate a broad potential of AI in not only" }, { "start": 1913.6, "end": 1929.2, "text": "creating content but also in making it more accessible and engaging for diverse audiences which is again crucial for building trust in official statistics and to do so one of the key objectives is to simplify technical" }, { "start": 1929.2, "end": 1944.679, "text": "jargon into plain language like I don't know if you use this prompt sometimes I do explain if I was five it's very useful so we need to simplify technical statistical jargon in the plain understandable language that is easy" }, { "start": 1944.679, "end": 1959.84, "text": "accessible to wider audience and to simplify the information navigations as example AI power chatbots can help bridge these gaps by providing a more intuitive and more userfriendly interface for answering questions like" }, { "start": 1959.84, "end": 1978.919, "text": "about data or metadata and numerous statistical organizations I exploring this approach according to our survey okay okay so um minding the timing I think we are approaching the conclusion of the presentation now and as we do so" }, { "start": 1978.919, "end": 1995.0, "text": "let's talk more about the current deliverables of our project as a again as a continuation of the white uh paper for the large language models so giving the rapid advancement of generating generative AI in the field uh" }, { "start": 1995.0, "end": 2010.84, "text": "we we like found out that unified Global approach is very important to fully understand the opportunities and limitations of these Technologies and although the final shape of the outcome is yet to be defined uh we are focusing" }, { "start": 2010.84, "end": 2027.039, "text": "now on creating a framework for sharing use cases comparing Ki applications and addressing strategic considerations such as risk management and AI policies we are also trying to develop a tailored governance framework to manage AI" }, { "start": 2027.039, "end": 2042.12, "text": "related risks including accuracy uh privacy and ethics and other risks that was mentioned earlier during the presentation so this Frameworks hopefully will guide statistical organizations in implementing AI both" }, { "start": 2042.12, "end": 2057.2, "text": "effectively and what is even more important responsibly and another key area of our focus is on understanding the foundations of prompt engineering and AI system architectures which involves emphasizing strategies for" }, { "start": 2057.2, "end": 2072.919, "text": "qualities assurance and selecting the right uh AI model based on scalability performance and cost Effectiveness because the number of AI models that is on the market now is progressing very fast and it's very difficult for" }, { "start": 2072.919, "end": 2086.919, "text": "statistical organization to Define uh which one would be the right suit for them so for all these efforts International collaboration and knowledge sharing are crucial as we navigate this evolving landscape so" }, { "start": 2086.919, "end": 2102.96, "text": "again many thanks for Alex and colleagues to organize this series of webinar which is very useful for that and from our side we are committed to creating an environment where best practices and Innovations can be shared" }, { "start": 2102.96, "end": 2118.68, "text": "openly uh so if you have uh any use case that could be interesting for us we would be very interesting to know more but if you don't have any use cases it's fine because despite of the growing collection of them the use of large" }, { "start": 2118.68, "end": 2134.119, "text": "language models and statistical organization is still in its infancy we can say and if you don't know where to start we recommend the following short-term actions first one is training so it's important for organizations to" }, { "start": 2134.119, "end": 2150.88, "text": "provide training across technical operational and managerial levels to raise awareness of large language models capabilities and limitations and how to use it responsibly of course next one is pilot projects so maybe it's uh useful" }, { "start": 2150.88, "end": 2167.0, "text": "to start with small pilot projects to explore large language models applications to build familiarity and for example to identify potential value before you scale it up and with open source opportunities that are available" }, { "start": 2167.0, "end": 2186.2, "text": "now you don't even need to start from scratch you can search what's open- sourced already and try to customize it based on your need um then develop an llm strategy so once uh you know uh how it's working once you aware and familiar" }, { "start": 2186.2, "end": 2201.319, "text": "with that uh it's a good step to create an overall strategy for large language model integration and last one but definitely not the least in this case it's to stay updated because since the public release" }, { "start": 2201.319, "end": 2221.8, "text": "of gpt3 um different large language model applications have multiplied progressively I try to find uh the chart I once find I once saw like maybe a year ago so I couldn't apologize uh there was chart that um there is a like line with" }, { "start": 2221.8, "end": 2239.599, "text": "the development of different Ai and Genera models which are progressively quite like steadily and slowly and then after 2021 when the gpt3 was released it's like going sharply up so um it's very important to devote continuous" }, { "start": 2239.599, "end": 2254.64, "text": "efforts to staying in the loop of technological developments and adapting these practices accordingly and uh with with uh that I want to thank you very much for your attention I'm happy um to answer any" }, { "start": 2254.64, "end": 2268.2, "text": "questions you may have or you can contact me via email if you're interested in joining the project or learning more about our initiatives with hlg MOS if you want to share your use cases or discuss them so thank you very" }, { "start": 2268.2, "end": 2285.48, "text": "much open for the questions thanks so much amelina um for for this great presentation um I mean I can also still remember when jet GPD was first launched and I tried it out at that time you still needed to get an" }, { "start": 2285.48, "end": 2301.44, "text": "invitation and so on and they wanted to limit of course a little bit the the use to save bandwidth and so on but it was really I would say an eye opener I thought okay this is going to be really cool and" }, { "start": 2301.44, "end": 2327.88, "text": "interesting and um yeah and then people started experimenting with it and so on um so I would like to open the floor now for questions and uh comments maybe I will also try to switch on the cameras and the microphones in the meantime I" }, { "start": 2327.88, "end": 2344.76, "text": "hope this will work um so if you have a question um please raise your hand and we will uh try to give you to the floor I'm trying to switch it on right now and I cannot see the hands right now because" }, { "start": 2344.76, "end": 2359.359, "text": "the maybe PA you can have a look are there any questions already ah now I can I I don't see Hands yet but maybe Emily take the two questions in the chat while we're trying to turn on the camera yeah exactly" }, { "start": 2359.359, "end": 2378.319, "text": "thanks so now uh I can see my screen again so maybe um I saw a question there for a colleague from eni from Mexico itel you want to try to come in I think you can unmute yourself now and put on the camera if you" }, { "start": 2378.319, "end": 2402.079, "text": "wish if you don't want to do that I can read out the question so I you want to come in okay can you he yes oh hi good morning guys well I have a question it was about the uh the policies that an organization" }, { "start": 2402.079, "end": 2421.079, "text": "needs needs to use for this generative AI sometimes I have noticed that not too many countries uh do have uh those policies or it's hard to to do them and my second part of the question is what are the most common challenges when when" }, { "start": 2421.079, "end": 2438.28, "text": "an organization is trying to develop these policies thank you thank you for the question yeah policies is a very interesting topic in this regard because it's something that needs to be developed from from scratch in a" }, { "start": 2438.28, "end": 2455.2, "text": "way and there is definitely a delay between the Practical Solutions and approaches organizations are uh exploring nowadays and the policies existing one of the reasons because um I know for European countries" }, { "start": 2455.2, "end": 2473.319, "text": "organizations were waiting for the release of uh European uh AI act so EU AI act and they were not uh pushing for developing something in house in terms of policies before the release of major document but now as it's been uh" }, { "start": 2473.319, "end": 2487.8, "text": "released uh like we see that the policies are being progressed and generally I would say that last several months we see more and more policies we're thinking about combining them also as a repository" }, { "start": 2487.8, "end": 2509.079, "text": "so we will keep you updated on this regard yeah thank you yeah I I think that's one one of the uh issues that are really popping up also in within our own office at the UN statistics division right where um with the appearance of" }, { "start": 2509.079, "end": 2527.44, "text": "the generative AI tools uh people were not really secure uh if they were allowed to use Ed them or how they could use them so there there are a lot of insecurities there and and clear guidance is not yet" }, { "start": 2527.44, "end": 2548.28, "text": "there um I guess one guidance that came out of our internal discussions is uh always check the outputs right uh when when you use generative a AI always uh check it for correctness and uh I think think that's one of the" }, { "start": 2548.28, "end": 2573.96, "text": "best advices one can give um there was another question um similar on on governance and regul Regulatory Frameworks by Omar om if you want to come in otherwise I can also read that one out Omar yeah please go ahead hello can" }, { "start": 2573.96, "end": 2592.839, "text": "can you hear me yes we can than I'm really sorry because I don't have the camera on my machine but thank you very much for this wonderful presentation amalina my question was yeah if you had any advice or Insight on the use" }, { "start": 2592.839, "end": 2607.76, "text": "potential use of AI in developing and strengthening uh governance and Regulatory framework for official statistics in countries that are uh really uh very lagging very behind on these issues and also uh in addition to" }, { "start": 2607.76, "end": 2623.64, "text": "that I may like that from your presentation uh my country in terms of of Statistics we really have a lot of gaps to fill and one of the major gaps is the gaps in Human Resources so we try to train people but it will take us many" }, { "start": 2623.64, "end": 2642.64, "text": "many years so I was wondering if in addition to the regulatory framework I think you mentioned some of them and we could also um uh like if if if AI could come in to feel but not really uh in like in know major way but at least some" }, { "start": 2642.64, "end": 2660.319, "text": "of these gaps thank you very much thank you very much it's a very important question you are asking and it's a very big concern I think uh it's something that's something that should be kept in mind that um even though" }, { "start": 2660.319, "end": 2678.96, "text": "generative fi sparked lots of interest interest Among Us generally it's uh just another new technology so in a way there is no new approach in handling this technology in terms of the gap between developing and developed" }, { "start": 2678.96, "end": 2695.52, "text": "countries um maybe um also worth mentioning that uh develop developed countries they're also learning now because there are no at least yet ready to go Solutions and with our uh projects we see like with the use" }, { "start": 2695.52, "end": 2712.48, "text": "case exploration we see that but uh like in a way uh countries are all going through the same process of trying to understand uh what is generative Ai and how it can be used for official statistics the advantage may be the" }, { "start": 2712.48, "end": 2732.4, "text": "developing countries are nowadays um be they are like we are all in the open source environment now so with the open source it's easier to get the access to different tools you don't need to implement them from scratch or develop" }, { "start": 2732.4, "end": 2749.24, "text": "them from scratch you can get ready to go solution from the open source you can get several of them and you can customize them based on your need so I would say that will be my answer for now and also you're very welcome to join us" }, { "start": 2749.24, "end": 2766.52, "text": "because we have not only developing countries we have developed countries as well joining our calls um maybe not that too actively contribute but to learn learn with us thank you uh yeah Omar if you're" }, { "start": 2766.52, "end": 2785.4, "text": "interested please maybe just put your email address in the chat so uh amelina can follow up with you later on uh let's take the next question so uh it's it's more technical question from saren saren uh uh can you please come in and also" }, { "start": 2785.4, "end": 2803.559, "text": "say uh for which institution you work yes hello everyone can you hear me yes please okay so my name is SAR and I'm from the Bureau of national statistics of Kazakhstan um so we are now also trying to adopt the AI tools for the" }, { "start": 2803.559, "end": 2821.48, "text": "usage in as a co-pilot also as a uh part of statistical production so we have already some results some experiments for example for classification of um unstructured data uh we we are using um ml models to" }, { "start": 2821.48, "end": 2841.4, "text": "classify the text from scanner data because it's like it doesn't have classification codes so we built the model that help us to classify the text from scanner data from fiscal receipts to uh to assign them the Code of product" }, { "start": 2841.4, "end": 2859.8, "text": "or services so this is one example another example we are trying also to use it as a board for our uh users of statistical information so we already have some results but it take it should um it requires much more computing power to" }, { "start": 2859.8, "end": 2877.48, "text": "fine tune further and what we are facing as a one one of the challenges for us is so as I said the computing power um and the one of the options is just to invest more like to to get that attract um additional uh budget" }, { "start": 2877.48, "end": 2893.44, "text": "but maybe I I was wondering if some organizations also face a similar challenge or maybe they have some agreement or collaboration maybe with uh big tech companies so that for the use of um statistical production they could" }, { "start": 2893.44, "end": 2913.24, "text": "have some maybe a little bit different mechanisms of collaboration so generally I was um interested in this uh question thank you thank you for the question and congratulations with your AI developments in Kazakhstan I think it's" }, { "start": 2913.24, "end": 2929.0, "text": "very impressive we would want to know more about your use cases in the detail but as for your question I'm not aware of any statistical organization having a specific partnership uh with big corporations in the field of generative" }, { "start": 2929.0, "end": 2945.599, "text": "AI uh from the survey we know that some organizations they mostly just extend the use of their subscription like for example majority of the organization they have like contracts with Microsoft and they can advance the" }, { "start": 2945.599, "end": 2966.28, "text": "use of Monica for example through which they can use chat GPT and so on um yeah so I'm I'm not aware of any additional contracts of the statistical organizations just for the use of generative AI yeah and also I I don't think we have" }, { "start": 2966.28, "end": 2988.44, "text": "uh the the cost Management in details because it's quite sensitive information from the statistical organizations okay thank you yeah thanks uh for that question let me take the next one in the chat Nicholas you want to come" }, { "start": 2991.079, "end": 2997.559, "text": "in okay so yeah please go ahead Nicholas" }, { "start": 3000.559, "end": 3019.28, "text": "question uh your your your volume is very low maybe you have the microphone can you try again yes it's here can you hear me yes now thank you thank you very much am now for this presentation the uh questions" }, { "start": 3019.28, "end": 3036.16, "text": "relates to the the use of uh of AI specifically for the production of uh of statistic rather than also I think internal use as this is well said also in the next comment so transforming the leads that you can generate through for" }, { "start": 3036.16, "end": 3055.079, "text": "instance a web scraping system into into more of an indicators statistics have you seen this kind of work as part of your mapping thank you oh thank you for the question your question really got me thinking I don't" }, { "start": 3055.079, "end": 3071.839, "text": "uh I cannot uh nav get you to the specific use case uh in mind yet but maybe you could check the repository we have I think on has done something similar but not exactly the same thing you're asking okay thank" }, { "start": 3071.839, "end": 3095.839, "text": "you yeah and and uh I wanted to add that um Nicholas you're working for the office of the UN High Commissioner for human rights that's correct right yes yeah yeah um then we have uh Kristoff Kristoff if you want to come in he works" }, { "start": 3095.839, "end": 3112.799, "text": "for the Regional Commission for Asia and the Pacific uh yes hi everybody uh I work for the Statistical Institute for Asia and the Pacific uh training institute so I was very happy that you mentioned training as something that ano should um" }, { "start": 3112.799, "end": 3131.4, "text": "should do in the future uh thank you for that Melina yeah my my question was more comment uh between the distinction of trying to uh use AI or L llm uh for internal use as a co-pilot something that helps and having something more" }, { "start": 3131.4, "end": 3147.28, "text": "automated more uh reliable more sustainable and more reproducible versus uh putting a chatboard that will answer to people and provide official statements or official statistics where the responsibility of" }, { "start": 3147.28, "end": 3165.16, "text": "the NSO would be uh somehow uh engaged so I wanted to have your views between the two use internal use and external use I think it was related to Nicolas question uh and somehow I would like to know your views on these two approaches" }, { "start": 3165.16, "end": 3184.64, "text": "on two use by NSO of AI again very interesting deep question yeah I would say I would say that there is a difference between the internal and external use especially taking into account that external use is" }, { "start": 3184.64, "end": 3202.4, "text": "in a way one of the application of you know what can be used internally as well but uh yeah uh probably like uh there are different challenges opportunities and risk but there are also similar one that um maybe with the external use" }, { "start": 3202.4, "end": 3220.76, "text": "there is extra extra pressure for for the organ ation to ensure that the output is as accurate and as coherent as it can be so yeah but I'm happy to follow up more and to talk more about this yeah probably the future will tell" }, { "start": 3220.76, "end": 3241.92, "text": "right on that yeah thank you so much yeah thanks uh Kristoff um so if there are additional questions uh just a reminder please raise your hand and then you can ask your question uh personally or if you want you can also type it into" }, { "start": 3241.92, "end": 3268.64, "text": "the chat um so uh maybe I have a question for amelina so this uh repository of use cases for generative AI um how does that work how can one uh add to this uh repository and and give you the information how would that" }, { "start": 3268.64, "end": 3285.92, "text": "work well you need to let us know what is the use case is about um like provide us with a detailed information so we could understand it and our readers would understand it as well and if it uh if it follows the criteria for putting" }, { "start": 3285.92, "end": 3301.76, "text": "the use case on the repository we will do it and we would be very grateful for helping us growing their repository yeah I think that's very helpful for people just to check uh if they have an idea of using generative AI" }, { "start": 3301.76, "end": 3317.52, "text": "they can already check okay has somebody done that before uh so I think that's really helpful so they would just write an email to you or you want to put the email address uh or if you have an Institutional email address just put it" }, { "start": 3317.52, "end": 3335.68, "text": "in the chat so people know where to WR to uh yes I will put my email in the chat now so I would be happy to follow up and talk more about what was discussed today but also M Leung put uh the very important comment that you can" }, { "start": 3335.68, "end": 3357.24, "text": "make a direct poll request uh in their link so oh if you want to compile a use case uhhuh that's great um then I have uh yeah one one more question was uh from the registration form if there are no other questions from the the floor" }, { "start": 3357.24, "end": 3377.16, "text": "right now um yeah I think we had a similar question already before but I just wanted to pick up on that uh once more so um the question is have large language models and AI pushed the Divide further apart between developing and" }, { "start": 3377.16, "end": 3394.96, "text": "developed statistical systems and the question is also how this can be addressed from data ownership privacy on computing power talent and infrastructure so what can be done to prevent that uh let's say the" }, { "start": 3394.96, "end": 3412.68, "text": "digital divide between developing and developed statistical systems grows yes and again uh in a way um I will repeat myself that uh there is nothing new with tackling this problem because it's just another another" }, { "start": 3412.68, "end": 3427.28, "text": "technical application that can spread to the Gap but I think think it also can be mitigated uh in the same manner and from our side what we can do as International Community uh we can make the knowledges" }, { "start": 3427.28, "end": 3447.839, "text": "and data accessible and so uh try to bridge these gaps with for example training data or um like as I said like uh code development application tools that can be open sourced and also invite um make U the doors of our projects open for all" }, { "start": 3447.839, "end": 3462.16, "text": "of the participants doesn't matter from what kind of uh cultural background and the level of technical development they are going and yeah probably in the nearest Future and we see we have Kristoff here provide extensive" }, { "start": 3462.16, "end": 3482.119, "text": "training so yeah that would be it thank you thank you uh Amina uh there was is also a question here in the chat not for you amelina but uh maybe just to point out that we will share the recording of This Global Network webinar uh on the" }, { "start": 3482.119, "end": 3501.799, "text": "global network of data officers and statisticians uh I will put the link into the chat where you can sign up if you're not yet a member um I also so we are approaching basically our the end of the this webinar now I just wanted to" }, { "start": 3501.799, "end": 3523.88, "text": "point out also apart from the possible increased gap between developed and developing countries there's also a possible new gender gap caused by generative Ai and we will have a Global Network webinar on that topic next month" }, { "start": 3523.88, "end": 3544.64, "text": "actually so uh we will have um two researchers uh from the University of Chicago and the University of Copenhagen who will be talking about this so please look out for the invitation and also for our next newsletter and you will find" }, { "start": 3544.64, "end": 3564.76, "text": "the information in there um so I think we can close now but before we do uh maybe everyone can switch on their cameras for a few seconds and also the microphones uh that would be really great and please join me" }, { "start": 3564.76, "end": 3583.319, "text": "uh to give a big round of applause for amelina for this very great presentation thank you thank you so much amelina thank you very much thank you thanks so much and uh in about two weeks we will have a Global Network" }, { "start": 3583.319, "end": 3603.039, "text": "webinar on Beyond GDP uh I will send out or we will send out the invitation uh and the information about that one shortly so thanks everyone for joining have a great rest of your day thank you thank you and thank you very much Amina" }, { "start": 3603.039, "end": 3612.16, "text": "again for for doing this presentation thanks thank you for organizing thank you for joining bye" } ] }