call_end

    • chevron_right

      Erlang Solutions: Erlang’s virtual machine, the BEAM

      news.movim.eu / PlanetJabber • 9 January • 5 minutes

    Welcome to the first chapter of the “Elixir, 7 Steps to Start Your Journey” series. In my previous post, I discussed my personal journey with the programming language.

    In this chapter, we will discuss the Erlang Virtual Machine, the BEAM.

    To understand why the Elixir programming language is so powerful and reliable, we must understand its foundations, which means talking about Erlang.

    Elixir runs on the Erlang Virtual Machine and inherits many of its virtues. In this post, you will learn a little about the history of Erlang, the objective with which it was initially created, and why it is fundamental for Elixir.

    What is Erlang?

    Erlang as a programming language

    Erlang is a programming language created in the mid-1980s by Joe Armstrong, Robert Virding, and Mike Williams at the Ericsson Computer Science Laboratory. Initially designed for telecommunications, it is now a general-purpose language. It was influenced by other programming languages, such as ML and Prolog, and was released as open-source in 1998.

    Erlang was designed with distributed, fault-tolerant, massively concurrent, and soft real-time systems in mind, making it an excellent choice for today’s systems. Most are looking for these features, in addition to having confidence in Erlang’s history in productive systems.

    Some of the characteristics of this programming language are:

    • It is a declarative language, which means it is based on the principle of describing what should be calculated instead of how .
    • Pattern matching is possible at a high level and also on bit sequences.
    • Functions in Erlang are first-class data.

    Erlang as the development ecosystem

    Up to this point, we have referred to Erlang as the programming language; however, it should be noted that Erlang can also refer to an entire development ecosystem that is made up of:

    • The Erlang programming language
    • The framework OTP
    • A series of tools and
    • The virtual machine, BEAM

    Erlang, as an ecosystem, was explicitly created to support highly available systems, which provide service even when errors or unexpected circumstances occur, and this is due to many of the characteristics of its virtual machine (VM).

    So, although Erlang as a programming language is pretty cool on its own, the real magic happens when all the ecosystem elements are combined: the programming language, libraries, OTP, and the virtual machine.

    Erlang's virtual machine, the BEAM OTP

    If you want to know more about the history of Erlang, the list of resources below will be very helpful.

    Resources

    Erlang Virtual Machine, BEAM

    The Erlang Virtual Machine, known as the BEAM , runs as an operating system process and is responsible for executing the Erlang code. It is also responsible for creating, scheduling, and managing Erlang processes, which are the fundamental basis of concurrency.

    Thanks to the BEAM schedulers, these processes can be executed in the most efficient way possible, allowing the system to be highly scalable . The processes do not share memory; they communicate through asynchronous message passing. This mechanism is the foundation for a system’s fault tolerance . As they are entirely isolated, the other system processes will not be affected if an error occurs in one of them.

    The BEAM is also responsible for parallelizing your concurrent Erlang programs, making the most of a machine’s resources. Initially, the virtual machine model was a single-run queue. However, it evolved into a run queue for each available processor, ensuring no bottlenecks and that Erlang programs work correctly on any system, regardless of the number of machine cores.

    Erlang Virtual Machine multicore

    Another characteristic is that storage management is automated. Garbage collection is implemented per process, which allows a system’s response time to always remain in the order of milliseconds without performance degradation.

    And lastly, one of my favourite features is error detection. The virtual machine provides all the elements necessary for efficient error detection and handling, thus promoting an always-available system regardless of failures.

    In summary, the BEAM is responsible for the scalability, distribution, and responsiveness of a system:

    • Manages the concurrency of it.
    • It has a mechanism for error detection and handling.
    • Make the most of the computer’s resources.

    If you’d like to learn more about the duo that is Erlang and Elixir, check out the “What is Elixir” post.

    Elixir in the BEAM

    Like Erlang, Elixir was also influenced by other programming languages, including Erlang itself. Its code runs on the Erlang Virtual Machine, which means it takes advantage of all its features and can use all the Erlang libraries and the OTP framework .

    Different programming languages ​​besides Elixir and Erlang run in the BEAM, but Elixir has ensured that the approach between BEAM and programmers is fluid and quickly understandable.

    Elixir code is compiled into bytecode that runs in the BEAM and is more compact than Erlang code. Its syntax is similar to how we communicate daily, allowing for early familiarization with the language, even if it is the first time you program with it. It also reduces the boilerplate and has amazing documentation.

    So, when writing code with Elixir, we have the best of both: a solid and battle-tested foundation that allows us to create fail-safe systems and, on the other hand, nice syntax, well-defined patterns, and code simplification, among other things. Thanks to this, Elixir has been so well accepted and has rapidly gained popularity.

    Elixir is a cool programming language that allows you to write code that is easy to understand and maintain and takes advantage of the Erlang concurrency model, which we will discuss in the next chapter.

    > iex
    
    
    iex(1)> list = [4,5,21,1,38]
    
    
    iex(2)> erlang_example = :lists.sort(list);
    [1, 4, 5, 21, 38]
    
    
    iex(3)> elixir_example = Enum.sort(list)
    [1, 4, 5, 21, 38]
    
    

    Example of how you can run Erlang and Elixir code in an interactive Elixir shell

    Next chapter

    In the next post, “Understanding Processes and Concurrency,” we will discuss how Erlang processes work and their importance in developing robust and scalable systems. We will also see how concurrency works in Erlang and how this relates to Elixir. Do not miss it! You can drop the team a message if you’d like to discuss Elixir in more detail.

    The post Erlang’s virtual machine, the BEAM appeared first on Erlang Solutions .

    • chevron_right

      ProcessOne: ejabberd 24.12

      news.movim.eu / PlanetJabber • 19 December, 2024 • 10 minutes

    ejabberd 24.12

    Here comes ejabberd 24.12, including a few improvements and bug fixes. This release comes a month and half after 24.10, with around 60 commits to the core repository alongside a few updates in dependencies.

    Release Highlights:

    Among them, the evacuate_kindly command is a new tool which gave the funny codename to this release. It lets you stop and rerun ejabberd without letting users reconnect to let you perform your maintenance task peacefully. So, this is not an emergency exit from ejabberd, but instead testimony that this releasing is paving the way for a lot of new cool stuff in 2025.

    Other contents:

    If you are upgrading from a previous version, there are no required changes in the SQL schemas, configuration or hooks. There are some Commands API v3 .

    Below is a detailed breakdown of the improvements and enhancements:

    XEP-0484: Fast Authentication Streamlining Tokens

    We added support for XEP-0484: Fast Authentication Streamlining Tokens . This allows clients to request time limited tokens from servers, which then can be later used for faster authentication by requiring less round trips. To enable this feature, you need to add mod_auth_fast module in modules section.

    Deprecation schedule for Erlang/OTP older than 25.0

    It is expected that around April 2025, GitHub Actions will remove Ubuntu 20 and it will not be possible to run automatically dynamic tests for ejabberd using Erlang/OTP older than 25.0.

    For that reason, the planned schedule is:

    • ejabberd 24.12

      • Usage of Erlang/OTP older than 25.0 is still supported, but discouraged
      • Anybody still using Erlang 24.3 down to 20.0 is encouraged to upgrade to a newer version. Erlang/OTP 25.0 and higher are supported. For instance, Erlang/OTP 26.3 is used for the binary installers and container images.
    • ejabberd 25.01 (or later)

      • Support for Erlang/OTP older than 25.0 is deprecated
      • Erlang requirement softly increased in configure.ac
      • Announce: no warranty ejabberd can compile, start or pass the Common Tests suite using Erlang/OTP older than 25.0
      • Provide instructions for anybody to manually re-enable it and run the tests
    • ejabberd 25.01+1 (or later)

      • Support for Erlang/OTP older than 25.0 is removed completely in the source code

    Commands API v3

    This ejabberd 24.12 release introduces ejabberd Commands API v3 because some commands have changed arguments and result formatting. You can continue using API v2; or you can update your API client to use API v3. Check the API Versions History .

    Some commands that accepted accounts or rooms as arguments, or returned JIDs, have changed their arguments and results names and format to be consistent with the other commands:

    • Arguments that refer to a user account are now named user and host
    • Arguments that refer to a MUC room are now named room and service
    • As seen, each argument is now only the local or server part, not the JID
    • On the other hand, results that refer to user account or MUC room are now the JID

    In practice, the commands that change in API v3 are:

    If you want to update ejabberd to 24.12, but prefer to continue using an old API version with mod_http_api , you can set this new option:

    modules:
      mod_http_api:
        default_version: 2
    

    Improvements in commands

    There are a few improvements in some commands:

    • create_rooms_file : Improved, now it supports vhosts with different config
    • evacuate_kindly : New command to kick users and prevent login ( #4309 )
    • join_cluster : Improved explanation: this returns immediately (since 5a34020, 24.06)
    • mod_muc_admin : Renamed arguments name to room for consistency, with backwards support (no need to update API clients)

    Use non-standard STUN port

    STUN via UDP can easily be abused for reflection/amplification DDoS attacks. Suggest a non-standard port to make it harder for attackers to discover the service in ejabberd.yml.example .

    Modern XMPP clients discover the port via XEP-0215, so there&aposs no advantage in sticking to the standard port.

    Disable the systemd watchdog by default

    Some users reported ejabberd being restarted by systemd due to missing watchdog pings despite the actual service operating just fine. So far, we weren&apost able to track down the issue, so we&aposll no longer enable the watchdog in our example service unit.

    Define macro as environment variable

    ejabberd allows you to define macros in the configuration file since version 13.10. This allows to define a value once at the beginning of the configuration file, and use that macro to setup options values several times during the file.

    Now it is possible to define the macro value as an environment variable. The environment variable name should be EJABBERD_MACRO_ + macro name .

    For example, if you configured in ejabberd.yml :

    define_macro:
      LOGLEVEL: 4
    
    loglevel: LOGLEVEL
    

    Now you can define (and overwrite) that macro definition when starting ejabberd. For example, if starting ejabberd in interactive mode:

    EJABBERD_MACRO_LOGLEVEL=5 make relive
    

    This is specially useful when using containers with slightly different values (different host, different port numbers...): instead of having a different configuration file for each container, now you can use a macro in your custom configuration file, and define different macro values as environment variable when starting each container. See some examples usages in CONTAINER&aposs composer examples

    Elixir modules for authentication

    ejabberd modules can be written in the Elixir programming language since ejabberd 15.02. And now, ejabberd authentication methods can also be written in Elixir!

    This means you can write a custom authentication method in Erlang or in Elixir, or write an external authentication script in any language you want.

    There&aposs an example authentication method in the lib/ directory. Place your custom authentication method in that directory, compile ejabberd, and configure it in ejabberd.yml :

    auth_method: &aposEjabberd.Auth.Example&apos
    

    For consistency with that file naming scheme, the old mod_presence_demo.ex has been renamed to mod_example.ex . Other minor changes were done on the Elixir example code.

    Redis now supports Unix Domain Socket

    Support for Unix Domain Socket was added to listener&aposs port option in ejabberd 20.07. And more recently, ejabberd 24.06 added support in sql_server when using MySQL or PostgreSQL.
    That feature is useful to improve performance and security when those programs are running on the same machine as ejabberd.

    Now the redis_server option also supports Unix Domain Socket.

    The syntax is similar to the other options, simply setup unix: followed with the full path to the socket file. For example:

    redis_server: "unix:/var/run/redis/redis.socket"
    

    Additionally, we took the opportunity to update from the wooga/eredis erlang library which hasn&apost been updated in the last six years, to the Nordix/eredis fork which is actively maintained.

    New evacuate_kindly command

    ejabberd has nowadays around 180 commands to perform many administrative tasks. Let&aposs review some of their usage cases:

    • Did you modify the configuration file? Reload the configuration file and apply its changes

    • Did you apply some patch to ejabberd source code? Compile and install it, and then update the module binary in memory

    • Did you update ejabberd-contrib specs, or improved your custom module in .ejabberd-module ? Call module_upgrade to compile and upgrade it into memory

    • Did you upgrade ejabberd, and that includes many changes? Compile and intall it, then restart ejabberd completely

    • Do you need to stop a production ejabberd which has users connected? stop_kindly the server, informing users and rooms

    • Do you want to stop ejabberd gracefully? Then simply stop it

    • Do you need to stop ejabberd immediately, without worrying about the users? You can halt ejabberd abruptly

    Now there is a new command, evacuate_kindly , useful when you need ejabberd running to perform some administrative task, but you don&apost want users connected while you perform those tasks.

    It stops port listeners to prevent new client or server connections, informs users and rooms, and waits a few seconds or minutes, then restarts ejabberd. However, when ejabberd is started again, the port listeners are stopped: this allows to perform administrative tasks, for example in the database, without having to worry about users.

    For example, assuming ejabberd is running and has users connected. First let&aposs evacuate all the users:

    ejabberdctl evacuate_kindly 60 \"The server will stop in one minute.\"
    

    Wait one minute, then ejabberd gets restarted with connections disabled.
    Now you can perform any administrative tasks that you need.
    Once everything is ready to accept user connections again, simply restart ejabberd:

    ejabberdctl restart
    

    Acknowledgments

    We would like to thank the contributions to the source code, documentation, and translation provided for this release by:

    And also to all the people contributing in the ejabberd chatroom, issue tracker...

    Improvements in ejabberd Business Edition

    Customers of the ejabberd Business Edition , in addition to all those improvements and bugfixes, also get support for Prometheus.

    Prometheus support

    Prometheus can now be used as a backend for mod_mon in addition to statsd, influxdb, influxdb2, datadog and dogstatsd.

    You can expose all mod_mon metrics to Prometheus by adding a http listener pointing to mod_prometheus , for example:

      -
        port: 5280
        module: ejabberd_http
        request_handlers:
          "/metrics": mod_prometheus
    

    You can then add a scrape config to Prometheus for ejabberd:

    scrape_configs:
      - job_name: "ejabberd"
        static_configs:
          - targets:
              - "ejabberd.domain.com:5280"
    

    You can also limit the metrics to a specific virtual host by adding it&aposs name to the path:

    scrape_configs:
      - job_name: "ejabberd"
        static_configs:
          - targets:
              - "ejabberd.domain.com:5280"
         metrics_path: /metrics/myvhost.domain.com
    

    Fix

    • PubSub: fix issue on get_item_name with p1db storage backend.

    ChangeLog

    This is a more detailed list of changes in this ejabberd release:

    Miscelanea

    • Elixir: support loading Elixir modules for auth ( #4315 )
    • Environment variables EJABBERD_MACRO to define macros
    • Fix problem starting ejabberd when first host uses SQL, other one mnesia
    • HTTP Websocket: Enable allow_unencrypted_sasl2 on websockets ( #4323 )
    • Relax checks for channels bindings for connections using external encryption
    • Redis: Add support for unix domain socket ( #4318 )
    • Redis: Use eredis 1.7.1 from Nordix when using mix/rebar3 and Erlang 21+
    • mod_auth_fast : New module with support XEP-0484: Fast Authentication Streamlining Tokens
    • mod_http_api : Fix crash when module not enabled (for example, in CT tests)
    • mod_http_api : New option default_version
    • mod_muc : Make rsm handling in disco items, correctly count skipped rooms
    • mod_offline : Only delete offline msgs when user has MAM enabled ( #4287 )
    • mod_priviled : Handle properly roster iq
    • mod_pubsub : Send notifications on PEP item retract
    • mod_s2s_bidi : Catch extra case in check for s2s bidi element
    • mod_scram_upgrade : Don&apost abort the upgrade
    • mod_shared_roster : The name of a new group is lowercased
    • mod_shared_roster : Get back support for groupid@vhost in displayed

    Commands API

    • Change arguments and result to consistent names (API v3)
    • create_rooms_file : Improve to support vhosts with different config
    • evacuate_kindly : New command to kick users and prevent login ( #4309 )
    • join_cluster : Explain that this returns immediately (since 5a34020, 24.06)
    • mod_muc_admin : Rename argument name to room for consistency

    Documentation

    • Fix some documentation syntax, add links to toplevel, modules and API
    • CONTAINER.md : Add kubernetes yaml examples to use with podman
    • SECURITY.md : Add security policy and reporting guidelines
    • ejabberd.service : Disable the systemd watchdog by default
    • ejabberd.yml.example : Use non-standard STUN port

    WebAdmin

    • Shared group names are case sensitive, use original case instead of lowercase
    • Use lowercase username and server authentication credentials
    • Fix calculation of node&aposs uptime days
    • Fix link to displayed group when it is from another vhost

    Full Changelog

    https://github.com/processone/ejabberd/compare/24.10...24.12

    ejabberd 24.12 download & feedback

    As usual, the release is tagged in the Git source code repository on GitHub .

    The source package and installers are available in ejabberd Downloads page. To check the *.asc signature files, see How to verify ProcessOne downloads integrity .

    For convenience, there are alternative download locations like the ejabberd DEB/RPM Packages Repository and the GitHub Release / Tags .

    The ecs container image is available in docker.io/ejabberd/ecs and ghcr.io/processone/ecs . The alternative ejabberd container image is available in ghcr.io/processone/ejabberd .

    If you consider that you&aposve found a bug, please search or fill a bug report on GitHub Issues .

    • chevron_right

      Erlang Solutions: Advent of Code 2024

      news.movim.eu / PlanetJabber • 4 December, 2024 • 3 minutes

    Welcome to Advent of Code 2024!

    Like every year, I start the challenge with the best attitude and love of being an Elixir programmer. Although I know that at some point, I will go to the “what is this? I hate it” phase, unlike other years, this time, I am committed to finishing Advent of Code and, more importantly, sharing it with you.

    I hope you enjoy this series of December posts, where we will discuss the approach for each exercise. But remember that it is not the only one, and the idea of ​​this initiative is to have a great time and share knowledge, so don’t forget to post your solutions and comments and tag us to continue the conversation.

    Let’s go for it!

    Day 1: Historian Hysteria

    Before starting any exercise, I suggest spending some time defining the structure that best fits the problem’s needs. If the structure is adequate, it will be easy to reuse it for the second part without further complications.

    In this case, the exercise itself describes lists as the input, so we can skip that step and instead consider which functions of the Enum or List modules can be helpful.

    We have this example input:

    3 4

    4 3

    2 5

    1 3

    3 9

    3   3

    The goal is to transform it into two separate lists and apply sorting, comparison, etc.

    List 1: [3, 4, 2, 1, 3, 3 ]

    List 2: [ 4, 3, 5, 3, 9, 3 ]

    Let’s define a function that reads a file with the input. Each line will initially be represented by a string, so use String . split to separate it at each line break.

     def get_input(path) do
       path
       |> File.read!()
       |> String.split("\n", trim: true)
     end
    
    
    ["3   4", "4   3", "2   5", "1   3", "3   9", "3   3"]
    

    We will still have each row represented by a string, but we can now modify this using the functions in the Enum module. Notice that the whitespace between characters is constant, and the pattern is that the first element should go into list one and the second element into list two. Use Enum.reduce to map the elements to the corresponding list and get the following output:


    %{
     first_list: [3, 3, 1, 2, 4, 3],
     second_list: [3, 9, 3, 5, 3, 4]
    }
    
    

    I’m using a map so that we can identify the lists and everything is clear. The function that creates them is as follows:

     @doc """
     This function takes a list where the elements are strings with two
     components separated by whitespace.
    
    
     Example: "3   4"
    
    
     It assigns the first element to list one and the second to list two,
     assuming both are numbers.
     """
     def define_separated_lists(input) do
       Enum.reduce(input, %{first_list: [], second_list: []}, fn row, map_with_lists ->
         [elem_first_list, elem_second_list] = String.split(row, "   ")
    
    
         %{
           first_list: [String.to_integer(elem_first_list) | map_with_lists.first_list],
           second_list: [String.to_integer(elem_second_list) | map_with_lists.second_list]
         }
       end)
     end
    

    Once we have this format, we can move on to the first part of the exercise.

    Part 1

    Use Enum.sort to sort the lists ascendingly and pass them to the Enum.zip_with function that will calculate the distance between the elements of both. Note that we are using abs to avoid negative values, and finally, Enum.reduce to sum all the distances.

    first_sorted_list = Enum.sort(first_list)
       second_sorted_list = Enum.sort(second_list)
    
    
       first_sorted_list
       |> Enum.zip_with(second_sorted_list, fn x, y -> abs(x-y) end)
       |> Enum.reduce(0, fn distance, acc -> distance + acc end)
    

    Part 2

    For the second part, you don’t need to sort the lists; use Enum. frequencies and Enum.reduce to get the multiplication of the elements.

     frequencies_second_list = Enum.frequencies(second_list)
    
    
       Enum.reduce(first_list, 0, fn elem, acc ->
         elem * Map.get(frequencies_second_list, elem, 0) + acc
       end)
    

    That’s it. As you can see, once we have a good structure, the corresponding module, in this case, Enum, makes the operations more straightforward, so it’s worth spending some time defining which input will make our life easier.

    You can see the full version of the exercise here .

    The post Advent of Code 2024 appeared first on Erlang Solutions .

    • chevron_right

      Ignite Realtime Blog: Florian, Dan and Dave Elected in the XSF!

      news.movim.eu / PlanetJabber • 21 November, 2024 • 1 minute

    In an annual vote, not one, not two, but three Ignite Realtime community members have been selected into leadership positions of the XMPP Standards Foundation! :partying_face:

    The XMPP Standards Foundation is an independent, nonprofit standards development organisation whose primary mission is to define open protocols for presence, instant messaging, and real-time communication and collaboration on top of the IETF’s Extensible Messaging and Presence Protocol (XMPP). Most of the projects that we’re maintaining in the Ignite Realtime community have a strong dependency on XMPP.

    The XSF Board of Directors, in which both @Flow and @dwd are elected, oversees the business affairs of the organisation. They are now in a position to make key decisions on the direction of XMPP technology and standards development, manage resources and partnerships to further the growth of the XMPP ecosystem and promote XMPP in the larger open-source and communications community, advocating for its adoption and use in various applications.

    The XMPP Council, in which @danc has been reelected, is the technical steering group that approves XMPP Extension Protocols. The Council is responsible for standards development and process management. With that, Dan is now on the forefront of new developments within the XMPP community!

    Congrats to you all, Dan, Dave and Florian!

    For other release announcements and news follow us on Mastodon or X

    1 post - 1 participant

    Read full topic